Ubuntu Hardware: Debugging Hard Problems

Some of the work done to enable Sandybridge Suspend (S3) and Hibernate(S4) showed how painful it can be to get hardware to do what it oughts to do! The problem arises when you find yourself with not many tools to debug what is going on, since your console and half of the OS functionality has already gone to sleep.

BIOS Vendors rely on the use of expensive JTAG debugging tools. While this is ok, it does not really allow for the community to participate and considerably increases the cost of enabling a system to work with Ubuntu.

Faced with this problem, the Hardware Enablement team at Canonical has set themselves the goal in Oneiric to create a “tool to analyze and suggest where suspend/resume is failing to help guide people through debug phase” i.e an automated version of Colin King.

The basic idea is going back to debugging basics: “Have you hit that print statement before dying?”. The problem is that you are trying to instrument a fairly complex part of the systems and you do not have a screen to print stuff to.

For the first problem, the team is trying an ready available open source solution: System tap. For the second problem, they are going old school: Audio and Light signals. Today most systems have speakers and a few LEDs to let you know one thousand irrelevant things that you can do with your keyboard. So why not put them to good use?

The blueprint goes beyond a simple “BEEP” when you hit your breakpoint :

We would need some hardware to record the lights/sound at a sensible speed:

  • Have another PC record the audio and interpret it
  • Leverage ham radio code already done to interpret sound

With some initial prototypes already floating around, I can’t wait to see what they deliver!

Going Agile: Scrum in a fully distributed team

Working with a fully distributed team has made me appreciate the beauty of having face time with your team!  Hence, I took the opportunity at UDS to get more acquainted with my colleagues.

Scrum by DarkMatter

As a first part to introducing Scrum to the team, we defined the high level goals (or Epics) for the 6 month release cycle. Part of what I have been trying to figure out is how to use the tools we have at-hand to get started. For the 6 months sprint backlog, we finally settle on launchpad blueprints. We are basically using a planning project within Launchpad, that will have a milestone per sprint/release. By prioritizing and assigning blueprints against the milestone, we get the backlog view.

Back at Symbian, we started by setting up daily scrums and weekly iteration backlogs. However, once the machine had started we struggled to define long term goals. It is hard to get out from the 2 week mindset.

Hence, with HW certification team at Canonical, I decided to prioritise the longer term goals. This was made very easy by the regular cadence of Ubuntu releases. The next step was introducing daily scrums and a 2 week iteration cadence within the 6 months sprints.

Are you standing up at the other end of the line?

With a fully distributed team, introducing regular formalised communication seems on paper an easy win. However, the trick is in the implementation. How do you do it? We decided not to have IRC meetings, based on previous experience. Eventually,  people did not read the comments from others and waited until their name pinged in the IRC channel to post a pre-baked update.  Another option was to Mumble our way through it! Continue reading “Going Agile: Scrum in a fully distributed team”