DNA sequencing: Creating personal stories

Data matters. A great example of a smart use of data is genetic sequencing. This involves 3 billion base pairs, although scientists only know what around 1% of these do. The arguably most important ones are to do with creating proteins. By looking at people with traits, diseases or ancestry, scientists have been able to pick out those sets of genes which seem to match with those attributes. For example, breast cancer risk is 5 times higher if you have a mutation in either of the tumour-suppressing BRCA1 and BRCA2 genes.

Due to science, there are now commercial providers of DNA sequencing available, such as 23andme. They market this as a way to discover more about your ancestry and any genetic health traits you might want to watch out for. To try this out, I bought a kit to see how they surfaced the data in an understandable way. The process itself is really easy, you just give them money and post a tube of your spit to them.

23andme.jpg
nice…

After a few weeks wait for them to process it, you can look at your results. Firstly, you have your actual genetic sequencing. This is perhaps really only of interest (or any use) to geneticists. As part of their service, 23andme pull out the “interesting” parts of the DNA which have been shown (through maths/biology) to correspond to particular traits or ancestry.

They separate this out into:

  • Health:
    • Genetic risks
    • Inherited conditions
    • Drug response
    • Traits (eg hair colour or lactose tolerance)
  • Ancestry:
    • Neanderthal composition
    • Global ancestry (together with a configurable level of “speculativeness”)
    • Family tree (to find relatives who have used the service too)

Part of what is smart about this service is that while it uses DNA as underlying data, it almost entirely hides this from the end user. Instead, they see the outcome for them. They have realised that people don’t care about a sequence like “agaaggttttagctcacctgacttaccgctggaatcgctgtttgatgacgt” but they do care about whether they have a higher risk of Alzheimer’s. Because some of these things are probabilistic, they also put a 1*-4* scale of “Confidence”: again this is easy to read at a glance. It isn’t very engaging, but it looks something like this:

Screen Shot 2016-09-26 at 16.18.08.png
Examples

Perhaps more visually interesting is the ancestry stuff. Apologies that my ancestry isn’t very exciting:

Screen Shot 2016-09-26 at 16.20.14.png
Ancestry, set to “standard” speculation levels (75% confidence)

 

I hope this has been interesting. Commercial DNA sequencing is a real success story not just for biochemistry and genetics, but also for the industrialisation of these processes and the mathematics and software that makes it possible. The thing that is especially cool, according to me at least, is the ability to make something as complex as genetics accessible, understandable and useful.

Proof: Little’s Law (why to Limit WIP)

Little’s Law states that:

The average number of customers in a queuing system = ( the rate at which customers enter the system ) x (the average time spent in the system)

Typically this might be applied to things like shoppers in a supermarket, but here we will focus on the application to software development. In a software development world, we often write it the same statement with different words, thinking about tasks:

Average Work in Progress = Average Throughput x Average Leadtime

Little’s law is beautifully general. It is “not influenced by the arrival process distribution, the service distribution, the service order, or practically anything else”[1]. This almost makes it self-evident, and since it is a mathematical theorem perhaps this is correct, since it is true in and of itself. Despite being so simple to describe, the simplest generalised proof that I have been able to find (and which we will not tackle here) is however trickier, since a solid grasp on limits and infinitesimals is required. Instead, we will consider a restricted case, suitable for most practical and management purposes, which is the same equation, with the condition that every task starts and finishes within the same finite time window. The mathematical way of saying this is that the system is empty at time t = 0 and time t = T, where 0<T<∞. A diagram to show this system might look something like this:

wip
Tasks all starting and finishing between t=0 and t=T

Proof

For our proof, we start with some definitions

n(t) = the number of items in the system at time t

N = the number of items that arrive between t = 0 and t = T

λ = the average arrival rate of items between t = 0 and t = T. The arrival rate is equal to the departure rate (sometimes called throughput), since the system is empty at the beginning and the end.

L = the average number of items in the system between t = 0 and t = T. This is sometimes called “average Work in Progress (WIP)”

W = the average time items spend in the system between t = 0 and t = T. This is called W as a shorthand for wait time, but in software development we might call this leadtime

A = area under n(t) between t = 0 and t =T. This is the sum of all the time every item has spent queuing.

Using this notation, Little’s law becomes

L = λ x W

which we will prove now. The following equations can be assembled from these definitions. We will need to use these to assemble Little’s Law.

  1. L = A/T (average number of items in the system = sum of time spent / total time)
  2. λ = N/T (average arrival rate = total number of items / total time, since every item leaves before t=T)
  3. W = A/N (average time in system = sum of all time spent / number of items)

We can now use these three equations to prove Little’s Law:

L = A/T                                                from (1)  

   = (A/T)x(N/N)                                since N/N = 1

   = (N/T)x(A/N)                                 by rearranging fractions

   = λ x W                                             from (2) and (3)

This is what we wanted, so the proof is complete.

What does this mean?

A trick to getting good outcomes from Little’s Law is understanding which system we want to understand.

If we consider our queuing system to be our software development team, our system is requirements coming in, then being worked on and finished. In this case, W is the development time, and each item is a feature or bug fix, say.

To have a quicker time to market, and to be able to respond to change more quickly, we would love for our so-called “cycle time” W to be lower. If the number of new features coming into our system is the same, then we can achieve that by lowering L, the average work in progress. This is part of why Kanban advocates “limiting work in progress”.

Alternatively, we can consider our queuing system to be the whole software requirement generation, delivery, testing and deployment cycle. In this case, we might have W being the time taken between a customer needing a software feature to it being used by them. By measuring this, we get a true picture of time to market (our new W, which is true measure of “lead time”), and we with some additional measurements we would be able to discover the true cost of the time spent delivering the feature (since our new A is means total time invested).

Outside of the development side of software, we can apply Little’s Law to support tickets. We can, for example, state how long a customer will on average have to wait for their query to be closed, by looking at the arrival rate of tickets and the number of items in the system. If there are on average 10 items in the queue and items arrive at 5 per hour, the average wait time will be 2 hours, since the rearrangement of Little’s Law to  L/λ = W gives us 10 / 5 = 2.

I hope that was interesting, if you would like me to explain the proof in the general case, let me know in the comments. I think it would be about 10 pages for me to explain, so in the spirit of lean I will only do this if there is a demand for it.

Anti-agile conferences

Let’s start by looking at what a conference is…

Conferences

Definition: “a formal meeting of people with a shared interest, typically one that takes place over several days.”

agileonthebeach2013.jpg
Agile On The Beach Conference (2013)

The features:

  • Topics shape the audience
  • Rigid program
  • You know what you are getting
  • Often high profile expert speakers
  • Accessible for old-school thinkers
  • Can be expensive for participants

 

Unconferences

Definition: “A loosely structured conference emphasizing the informal exchange of information and ideas between participants, rather than following a conventionally structured programme of events.”

unconference
Manchester BarCamp 7 (2016)

The features:

  • Audience shape the topics
  • Flexible (unknown) program
  • Lean
    • Not so much preparation for speakers
    • Law of two feet: If a session is not useful to someone they will leave, which is less wasteful
  • Variable quality speakers
  • Everyone participates, which is more effective for learning
  • Often cheap/free for participants

Which is more agile?

I will now attempt to show that agile conferences are a paradox. I know it may seem odd applying agile (for software development) to something like a group of people communicating, but let’s give it a go. Perhaps we can compare some traits of Conferences and Unconferences to the Agile Manifesto:

  • Individuals and interactions over processes and tools
    • Unconference is more agile, since unconferences are attendee-led
  • Working software over comprehensive documentation
    • Arguably both styles similar with an Unconference perhaps slightly better, since conferences often document lots in their preparation
  • Customer collaboration over contract negotiation
    • Unconference is certainly better here, since unconferences are attendee-led (and the law-of-two feet means that “customers” don’t get what they don’t want)
  • Responding to change over following a plan
    • Unconference is massively better here

I put it to you that as agile evangelists, we shouldn’t be encouraging traditional conferences, which are prescriptive, irresponsive, contractual and process-driven.

Instead, we should support unconferences, perhaps by organising, participating or sponsoring them.

 

Pilates changed the way I run

I tried a Pilates video a few weeks ago. I know that might not be very manly, but I thought I would try something different to shake things up. I hoped it would be a bit like yoga, and help with my core strength.

It had an unexpected benefit… I discovered Lateral Breathing. This involves breathing deeply and engaging core muscles, in order to maximise oxygen exchange. I’d strongly recommend it, it feels great! There are apparently lots of benefits, and the main one I’ve noticed is to do with running. And it isn’t just in the way it feels (since I track everything I have data):

  • Over the same course…
  • My heart-rate is very similar…
  • In exchange for a 1% higher cadence…
  • And 30secs/mile faster speed.
  • This is much more than the improvement I usually see in a similar 2 weeks of training.

So yeah, try it!

guyrun
Smoggy running in Chongqing last year

You might be wondering why I’m talking about this. Well, it feels like innovation. You might’ve read last week’s post about Active Procrastination: you never know what might help you approach a problem in a different way.

Innovation often occurs at the crossings between two disciplines. Some varied examples:

So I suppose this blog post is a little nudge for encourgaging doing something different, outside your normal zones, in order to experiment and learn.

Go on, do something weird. You never know what ideas it might spark.

 

 

 

 

Introducing… Active Procrastination!

I think this is one of the first times I’ve created something new in a psychology context. Let me first outline the problem. I struggle to relax and unwind, but sometimes (even/especially during the workday) I need to do something different to get away from stuff. I guess you’d call this procrastination. Often I would go for a walk or do some exercise, and my friends were saying why don’t you just watch TV / read the internet / play flash games. But I couldn’t. If I did, I just felt guilty and certainly didn’t get pleasure or relaxation from it.

Research by Chu and Choi suggests that:

Not all procrastination behaviors either are harmful or lead to negative consequences.

I wanted to be able to procrastinate, not least because of this cool research that can be seen in this wonderful article. Basically, people who were told a problem, then played minesweeper, then attempted to answer a creative problem, came out with many more answers. As mentioned in the article, this may not be strictly speaking procrastination, but what I can do is replicate this creativity study environment at home for myself in the hope of enjoying their reported 28% increased creativity.

Cue the world debut of the Active Procrastination Board:

img_20160919_131453

Each card has something different on it, a little like Brian Eno’s Oblique Strategies. Unlike oblique strategies, they are all activities designed to:

  • Take me away from what I am doing
  • Take varying lengths of time (so I can choose what I need)
  • Be things I would like to do
  • Be playful
  • Be a large range of different types of thing
  • Be mostly “wholesome” activities
  • Not be too intellectually or physically taxing

When I first used this, I started having it with dice to pick the card, one for the row-number and one for the column-number. However, I found I wanted to use the board more if I picked one I wanted to do at that time instead. For example, “heading to the library to read a random book” in the rain is annoying and takes ages, but in dry weather is fine.

Please let me know your thoughts, and if you try creating your own I’d love to hear about it!

 

Deep Dive: Retrospectives

The term “retrospective” is prevalent for any of the Continual Improvement meetings, largely due to the popularity of Scrum. However, this post really refers to any meeting where a change to process is made. Whatever you might call this, it is very important, as I hope I will show in this post.

Why should the team do a Retrospective?

  1. It allows them to talk about their feelings in a safe space (Social Connection/Relatedness)
  2. It allows them to make small changes to their processes, to try to make their delivery better (Mastery/Competence)
  3. It allows them to feel in control (Autonomy)
  4. It allows them to step back and see what they’ve achieved (Purpose/Competence)

So a Retrospective gives a lot of the things that Self-Determination Theory holds are important for motivation (highlighted in bold in the list above). If you aren’t doing retrospectives in some form, then you are perhaps missing out on a chance to motivate and inspire the team, as well as not helping them to deliver more.

Who should run it?

There are a few options here:

  1. An external (to the team) facilitator. This has the advantage of allowing each team member to equally participate. The disadvantage is that the team members may feel that the meeting is happening to them, rather than for them. And of course someone needs to arrange the facilitator.
  2. A scrum master or similar. Advantages include that they are probably good at facilitating, and know how to help the team work well together, and that it is their role so they will make sure the meeting is organised. The main disadvantage is that it can dis-empower the  team (as above).
  3. A team member. This has the advantage of making team members feel engaged with the process (especially if they each take a turn). It has the disadvantage of needing meta-coordination, and can lead to variable quality.
  4. A manager. I think this is a bad idea as it reinforces the hierarchical ideas, unless the manager is a very effective Servant-Leader and able to keep her/his opinions to a minimum!

Who should be there?

Everyone interested in the team performing well. So definitely the team including Product Owner (or equivalent), and anyone else who might having valuable insights to share.

What should the outcome be?

The outcome of a Retrospective should be a very small number of SMART actions. A small number since the team can be considered a Complex Adaptive System, and as such according to Cynefin (and common sense) should be conducting small experiments rather than many large changes. By SMART, I mean Specific, Measurable, Achievable, Relevant, and Time-based. Sometimes these actions will be about how the team work together, other times they might be about improving the working environment, software, tools, or interactions with others. Whatever is important to the team at the time.

It is also very important that a Retrospective is FUN! The  desired outcome here is that the team want to do it again.

What should happen in a Retrospective?

I’d describe a good Retrospective as Serious play. By that I mean playful but about something serious, with serious outcomes. Tools like retromat give great ideas for how to construct these kind of playful sessions. In particular, using colourful postit-notes, images, memes and other creative tools can make the session feel more fun. For me the important parts are:

  • Set the scene and let people check in. It is important here to get the energy levels up and frame this positively.
  • Write down separately comments about the last Time Period in some form
  • Allow people to speak about them
  • Agree on important core issue
  • Get the team to come up with and commit to a SMART solution or experiment
  • Get feedback on the retrospective itself

I think it is important to allow people to separately write things down because it allows the less vocal in the team to say what they feel without interruption by the more extrovert. It also keeps the discussion wide initially, before narrowing. This intentionally mirrors the Explore-Exploit of “the bandit problem” which is often applied to innovation theory. Essentially with Retrospectives we are helping the team to innovate themselves.

How often should we do Retrospectives?

Scrum says once a Sprint, where a Sprint is usually between one and three weeks. However, the team should decide the frequency. More frequent Retrospectives give fast feedback, and shorter feedback loops, so if you are very “agile” then it is likely you will be doing this very regularly. I’ve worked in teams that have a Retrospective each week, and others that have a Retrospective only once month.And of course they can change the frequency whenever they need, through the Retrospective itself. It really depends on what the team wants (more often is better as it gives faster learning).

 

Working from home?

I’m often asked “How do you manage to ACTUALLY work from home so often?”.

Well, for me the answer is simple: routine and adapting.

Thanks to reading lots of psychology/sociology books, I also think I have a reasonable handle on what makes humans happy: things like Social Connectedness for example. That means I build them into my rituals.

My schedule:

6AM: Wake up. Yes, every day same time. Apparently this is better for you. I also get out bed immediately, no snooze-button. I have always been a morning person, and appreciate this doesn’t work for everyone. My wife is now almost adjusted to this schedule (we find sticking to it works much better if we both want to sleep at the same time).

6:05AM: Go exercise! I follow a training plan as I’m normally on autopilot this time in the morning, and I find it keeps me motivated. This can be anything from a 30 minute light cycle to a 2hr run. Currently I’m following this half marathon plan. On my way out I let out the chickens (and if everyone else has bins out, I do that too. I can lose track of what day it is).

 

IMG_20160824_110402
7 days behind… I can’t keep up.

After this, I stretch, have a protein shake and
shower, then read the news over breakfast (which invariably involves eggs).

 

9AM: Depending on how long the workout has been, this might be a hour or on a rest day even two hours earlier: but I will definitely have started work by 9AM. First up I write down the stuff I’ve thought of during my workout – I often am like “oh yeah I must do XX”, for example today as I am about to go on holiday it was to look up the best way of exchanging Czech Republic Koruna. I add these as kanban cards to my board. I then check the priorities for the day look right: if I’m lucky I can catch my wife before she goes to work and see if she has anything she wants my help with. An important part of this process is that I’m blending “work” and “home” things together as part of the same list.

IMG_20160824_110025
Today’s board. The high WIP is due to waiting for replies to emails.

This allows me to flex onto whatever is more important, reducing the amount of “work” I do for “home” if that is what is needed. These tasks might me anything from “Do laundry” to “Write a blog post” to “Find out if data.gov has usable, open Littering Data”: I think the variety of these tasks keeps me engaged throughout the day.

 

9:15AMish: Crack on! Picking up either what was in progress yesterday, or whatever is more urgent today, I do it. And repeat. During these large work periods, I break it up with water breaks and whenever my Garmin Vivoactive watch tells me I need to get up and go for a quick walk round the block (around every hour).

12AM: Early lunch. No resting, just refueling. Usually leftovers or steamed veg.

12:30PM: Back to work. At around 3-4PM I am likely to start flagging, so I’ll probably change what I’m working on. I’ll also try to speak to a friend or family at some point.

6PM: Stop work. Sometimes if I’m not feeling in the mood this will be earlier. Make dinner. Eat with my wife (usually not chatting as she doesn’t like “eating air”).

6:30PM: Go for a walk with my wife (usually 5km, in the countryside). We talk about our days etc.

8PM: Home, get ready for bed. Read (currently”A song of ice and fire” and “Chase one rabbit“).

10PM: Lights out.

You might be thinking that anything like “going out” or requiring a late night would throw everything out of whack: and it does. I try to see friends on nights before I’m having a rest day for exercise, and I allow myself to wake up later if I need on that day. I ran several experiments where I reduced sleep, and it led to much less being done during that day. Seeing people is really important, and as such I try to go to one work-related event (eg Agile Meetup) and one friend-related event (eg pub) a week.

I make sure I top-up my social connectedness at weekends: like many young professionals I don’t have “free weekends” very often. The biggest thing I miss about working in an office is the social connection I found with colleagues, and indeed I am consider going to a co-working space, or even choosing to take on a full-time contract that would require me being in an office for several months.

I am also not fixed in this way of working, I am always trying new things. I tried a month of 5am starts in July, and did evening work sessions in August. Again, Inspect and Adapt is king.

 

 

Agile research needs help

Reading some academic papers on Agile recently, there is a huge misunderstanding about what Agile means.

In particular, some academics use the word Agile when really they mean Scrum. There are papers (for example this one) which say “agile teams” and continue on to describe how long their iterations (or even worse, Sprints) are. Sprints are a feature of Scrum alone, of all the Agile frameworks: the Agile framework of Kanban for example does not have iterations. Maybe this is an accidental slip-up, but I would hope for more accuracy from academics.

There are also lots of really great articles by people who helped come up with the very idea of Agile. The reason I am worried about even a few dodgy papers is that one experiment (or research paper) can easily be cited out of context or generalised by another paper, leading to huge misconceptions and ultimately a misleading (but popular) HBR or linkedin article without the right context.

Having said this, it is great that academics are starting to look at Agile: some claim the “research lags years behind of the practice” [1]. For me in particular, I am interested in whether the motivation experiments (which we rely upon for incentivising software developers and trying to keep them motivated) truly apply to software development tasks. This BBC article summarises some of the studies, on playing computer games, maths tests, and repetitive-key pressing. But there are none that I can find which examine software development motivation.

In some disciplines, generalising and abstracting is necessarily bread-and-butter: in Physics this makes loads of sense, but in Psychology given how complex and counter-intuitive some human decision making can be (eg “it was found that everyday hassles and uplifts were a better predictor of concurrent and subsequent psychological symptoms than were the life events scores” [2]). I’d love to share more of these things in a subsequent post, though I am by no means an expert in Behavioural Economics.

2014 most common jobs
The most common job in each US State (2014)

The most common job in several US states is now Software Developer (rather than Secretary or Farmer, as it once was) [3]. Given the huge numbers of people now employed as software developers throughout the world, it would be great to be sure that we are interacting and incentivising them properly.

I hope this post inspires some academics to get more involved in fieldwork and talking with industry, and also inspires some Agile evangelists working in organisations to take a bit more of a rigourous and self-critical perspective when it comes to documenting and sharing what they are learning.