Here is my short, simple step-by-step guide for smart collection of data.
Step 1) Determine what matters, ideally in accordance with a Company or Product vision
Step 2) Come up with as many different ways of measuring aspects that matter or impact what matters
Step 3) Collect data! Ideally setting up easily repeatable ways of this, and automated wherever possible
Step 4) Form hypotheses: how do you believe certain measures affect your vision? What do you expect the data to tell you?
Step 5) Collect more data
Step 6) Test your hypotheses
Step 7) Collect even more data. Quite simply, the more data the better.
Let’s look at an example. Suppose a government manager wishes to improve the innovation of her employees.
Step 1) Target: what matters here is “innovation” – which we define more precisely in…
Step 2) Measurement: Some of the ways in which innovation can be measured are volume of ideas, number of staff submitting ideas, percentage of staff submitting ideas, value delivered, employee perception of innovation produced, manager perception, and customer perception (in this case the public would be the customer), etc.
Step 3) Collection: This involves ensuring that things are centrally recorded and surveys are done to create a baseline.
Step 4) Hypothesis: It is suggested that an innovation rewards’ ceremony would help to improve the morale. Note that it is important that the hypothesis is formed after the first data collection, as we want to be able to dig deeper into anything interesting we find. This means that often we need to collect more detailed data more specifically targeted towards proving or disproving our hypothesis.
Step 5) Collection: A more accurate, probably quantative, measure of morale is added to the existing survey.
Step 6) Action: An innovation rewards’ ceremony is run.
Step 7) Collection: The survey is conducted again – morale is measured as having improved. Success! Note that the other measures (e.g. the volume of ideas produced) are now also being consistently measured and can be easily tracked throughout future experiments.
After running through these steps we can ask ourselves the following questions.
What do we now know?
- Key measures, and how they are changing with time
- Whether the key measures remain the same, or if other aspects should be considered.
What can we not imply?
- “Correlation does not imply causation”: just because a trend becomes apparent this does not mean that one workplace modification is the main contributor to a measured difference. For example, if morale improves during the summer months this may have been due to nicer, warmer weather rather than any particular managerial decisions.
- We cannot assume that any trends apply in similar cases elsewhere: our sample is too small and too specific. Luckily, a full research paper is not the goal here!
As some of you may have noticed, this is very similar to the Six Sigma methodology of “Define, Measure, Analyse, Improve, Control”. It also mirrors the “Plan, Do, Check, Act” process found in many management handbooks.
The detail of the steps you yourself follow is not particularly important here, all I am really suggesting is to:
- Ensure you are working on what really matters.
- Add wider data collection before directing all your attention to one particular area. This way you will have a richer understanding of the problems and opportunities.
FUZA 
I am an unusual analyst. I find that many miss the step of understanding the reality in a real tangible way. The three gen philosophy is important.
LikeLike