Today we had a lunchtime talk about JavaSpaces, which is a form of grid computing (this is a new thing for me, so if I get anything wrong, yell).
Grid computing is when a lot of computers are used to split up processing of a hard task. The SETI screensaver uses your CPU at home to form a big grid. If you want to draw a Mandlebrot fractal, you can parcel out the various bits to nodes in a grid to calculate (this is so easy that everyone uses it as an example). There are lots of ways in which grids can be implemented, and some implementations are better than others. Some of them push tasks out from an orchestrating machine. Others have nodes which pull their tasks out of memory space.
JavaSpaces uses this latter technique, and also shares the memory space amongst several different computers. While we were learning about this, we asked a lot of questions and began to form a picture of how a really good, robust grid would work.
- A good grid has redundancy in its memory store, so that if one node goes down another has the necessary information to continue.
- Some really good grids have nodes which work out when they’re being polled for information too often, and find another node which isn’t being used so much to share it with.
- A good grid will work out when a given task is killing nodes too often, and flag the task as something which needs fixing.
- A good grid will look for high-priority tasks to run first.
- A good grid runs transactionally, so that if a node fails to complete a task, it doesn’t break the grid. Instead the task stays on the list.
- Some really good grids accept that their nodes aren’t perfect (for instance, floating point arithmetic may differ from one hardware base to another). These grids use a voting system, so that the result of one machine is checked by others.
Can you see where I’m going with this?
There are a lot of parallels with Agile Development and collaboration between Agile developers.
- A good team will communicate, so that everyone knows what’s going on.
- On a good team, developers are encouraged to share information instead of hoarding knowledge. This reduces the bus factor.
- A good team will spot problems which kill their velocity and sort them out.
- A good team will take on the tasks which the customer has decided have the highest business value.
- On a good team, members won’t check in stuff that breaks the build. They’ll use version control so that if they do accidentally do something which breaks the build, they can roll back their changes and try again.
- On a good team, developers ask for help and accept it from other developers. They will pair together, and discuss possible solutions until they have an idea of which solution is best.
I wonder – is Agile naturally suited to this kind of development work? Does splitting things into manageable chunks lead to easier, more even development? Would developers communicate better if we were coding for a distributed system? If we developers lend out our brains, will we discover life beyond our world?