Wednesday, August 3, 2011

Random Notes: Thoughts on metrics for engineering tools

  • If training is required for successful usage, what is the average half-life of the training. In other words, if a group of users is trained, how long until 50% of the users will forget some key aspect of tool usage which drives them to abandon the tool?
  • What is the average time for user to need to go to the help files to complete a task if they do not use the tool constantly?
  • Can a user successfully use the tool without training?
  • Are the documentation and examples sufficient for self learning?
  • How many actions are required to complete a ‘quickstart’ example?
  • How many decisions are required to complete a ‘quickstart’ example?
  • How many choices are the in each decision in a typical workflow?
  • How difficult is it to integrate the tool into automated work flows?
  • How difficult is it to customize the tool?
    • Can a power user customize the tool?
  • How long does it take to introduce a new feature in the tool?
  • How many sentences does it take to describe why a user should adopt the tool?
  • In the absence of process enforcement, would the users naturally adopt this solution?
  • What is the time saving for the individual, team, and organization from the adoption of the tool?
  • If the tool reduces error rates, is there feedback to the users to help them understand the improvement?
  • Can the input and output to the tool be reused so that the effort can be reapplied?
  • What is the ‘activation potential’ to get a new user to adopt the tool?
    • Do new users request access to the tool?
  • In a corporate setting, how difficult are the permissions to manage?
    • If a new user if not setup, will the team be able to duplicate the permissions with without calling the developers?

This work is licensed under a Creative Commons Attribution By license.

No comments:

Post a Comment