So what took so long? well mostly a lack of time, work keeps me very busy during the week, and I got a little carried away learning some android development.
So progress on the wine rack has come in fits and starts. Most recently it was effectively done,painted and sanded, but awaiting the lazy susan bearing required t make it spin. When this finally arrived I realised that I needed a reasonable size ‘foot’ piece for the bearing to provide balance and stability, with heavy wine bottles there was the potential for the whole thing to tip over if unevenly balanced.

A suggestion was made that I simply needed to make sure we always consume wine in multiples of 2 bottles, to ensure even distribution at all times. However I figured a little extra time to make a good foot for it would allow things to be at least a little one sided without tipping.

The completion of this project was also timed with my awesome wife returning from a trip to France, she brought back a couple of cases of wine to help fill up the wine rack

Overall I enjoyed this project and I’m pleased with the result. I’m sure I could have constructed a more simple wine rack with a16 bottle capacity that would have fit in the space, but this was just a lot more fun and I feel like the space is more interestingly used .

So when I’m learning a new development language or library, I mostly look for an example I can pull apart and restructure to suit what I’m after. In that regard the Android development instructions available are really very good, and I enjoy the style. Mostly there are pages that give a fairly high level idea of the kinds of objects and usage patterns you need to use for various things, then link to examples that show them being used.

In my case I’m picking through the LunarLander example to get an idea of how to work with SurfaceViews and do some custom rendering.

As helpful as that has been, I have found myself getting very familiar with one dialog pop-up in particular.

Unfortunately has stopped : OK

As a user I guess this is about as much as you are interested in knowing. As a developer I kind of wish there was a button that would take me of the stack trace of what went wrong. So far I have not investigated the ‘right’ way to debug such errors. However I do have a way.

The first thing I tried when looking at android development was the rather awesome AIDE, a whole android development environment that runs on an android device! I loved that I could create a sample app, start hacking it around, hit ‘run’ and be running the new app in a few seconds right there. In someways it was AIDE that rekindled my interest in developing for mobile by reminding me that it really could be quite easy to get going. The fact that I wound up being drawn into a much more ‘serious’ development environment of eclipse/git/jenkins was mostly because AIDE gave me a chance to think about some ideas and decide I had one I really wanted to work on.

In my playing with AIDE I found it had an option to ‘view LogCat’ which basically shows a tail of the system log. The first thing I noticed in this was that there are errors happening in software under the hood all the time. The second was that I could see the errors of my own app.
So now, I know I can trigger an error in my app, switch to AIDE and normally catch the error in the log. There is an issue that if you’re not fast then the error disappears as the log buffer isn’t that long. However, so far it’s been all I need to get a glimpse of the underlying exception that has lead to my downfall.

So far all my issues amount to figuring out the life-cycle of an android app and the state transitions I’m responsible for handling. Learning the right ways to ensure that state is persisted and restored at all the right points, hopefully without massively repeatedly doing calculations when they are not required.

As I write this my problem appears to be when flipping back to my running app after switching away. The calls to thread.start occurring again against an already running thread. At this point I wonder if I should have actually compiled up and installed the lunar lander example to check that it doesn’t not suffer the same failings, since I’m increasingly unable to figure out what my app is doing differently in this regard and getting into new code to guard the appropriate conditions.

Despite this on going issue with handling the application life cycle, I have thus far been very happy with my rate of progress. I had some fun rendering a QRCode from a bitmatrix. After briefly wondering if an existing library existed to perform the rendering, I realised that with the hard part done, eg some data encoded into a bitmatrix, rendering it out was actually trivial (as attested by the fact that my code to do so work first time). Now I’m playing with how transparent I can make them whilst still scanning reliably, and I may play with whether colour has any effect.
For the moment I’m happy to use a standard qr code reader to test with, but I think I will also play with integrating one myself, since I think it will be good experience.

My biggest problem with developing an android app turns out not to be the environment or the instructions, it is that the UK is experiencing something of a mini early summer. The weather has been so nice that between work and wanting to make the most of sunny weekends, I really don’t feel I should be spending time indoors at the computer more than I already do.

UPDATE: I did some searching and found others had the same problem using the LunarLander example, it has a bug in that it is not valid to call start on the thread again after the first time. the solution is to ensure the app exits on stop and therefore cleanly restarts when you switch back to it.

However after a while I grew tired of the n900 as a phone, whilst I loved the potential of the platform, it felt like nokia didn’t really care about it, and Android was very much the new hotness. So much more support, more apps, and generally less buggy. And so I handed over the reins to Witter development to some guys in the community that were interested in continuing to support it and I moved on. Having just had a quick look I was quite amazed to see that witter has has 318k downloads. Not bad for what started as little more than an coding exercise.

Back when I was developing witter I wrote about setting up a development environment for maemo, having just googled I found it’s still in the first page of results, so hopefully it has been helpful to others, and with any luck this will be to.

What’s taken me so long to get around to developing again? Well I’d say the issue is two-fold, firstly I got a new job about 16 months ago and it keeps me very busy. Secondly, there are already a bazillion apps  for android. I just didn’t find the time to come up with any ideas that inspired me to get going. With little free time, it is easier to spend it on other things. (mostly wood work projects)

But now I have an idea, and so I started looking at what I wanted the development environment to be like. Android being Java means the IDE of choice is still eclipse, and this time I don’t have to try very hard to get things going (I recall getting the python tooling installed in eclipse was a pain). Google have made it super easy to just pickup the android developer tools update site, download and you’re ready to rock. But there is more to a dev environment than the IDE.

In the last year at work I’ve been using mercurial for source code and team city for continuous integration server. This is a ways on from my tool chain of maemo days, I don’t think I’d though much about using continuous integration for my personal projects. I was using SVN, but mostly that felt like a chore to make sure I had version control rather than an integrated part of the process.

In order to spread my experience a little, I decided to try out GIT for source and Jenkins for a CI. I did some research and found that Jenkins has an android plugin that allowed it to do clever things with obtaining the android libs for compilation etc. and GIT integrated with both Jenkins and eclipse via plugins.

The machines I have access to at home amount to my transformer prime (which is amazing but not really suited to this kind of task), my Asrock ION 330 which runs my media centre, a netbook. and an old imac g5(ppc). there is also a main computer which my wife uses for her work, so that wasn’t really in the picture as any time I have for coding she would be spending writing. When I developed witter, I mostly used my work laptop. However it was never and ideal situation, for various reasons I’d rather keep my own projects separated completely from work. Mainly if I’m on my work laptop I’m probably logged into work and therefore distracted by work.

So I decided that I would install Jenkins and a main git repository on my media centre pc (it’s on all the time anyway) and install eclipse and the android developer tools on the mac. I’d not used the mac at all for a few years, it’s spent at least the last year or so packed up in its box. So I got it out, cleaned the keyboard which was quite grimy, and fired it up. Within short order I found that this being an old PPC mac, there is no download for eclipse on ppc mac. and no android developer tools either. In fact this is now a decidedly dead end of technology with very little support for it. However, ubuntu do have a ppc build, and so I figured I had nothing to lose in wiping the old os X and sticking a recent ubuntu on.

In the meantime I installed jenkins, then I had to figure out how to switch its default port to something else  (I already have a few web type things running on the machine so there was contention for jenkins default 8080 port). The instructions I could find appeared to point to the wrong place. in the end I wound up navigating into /etc and then:
find . -name “jenkins*”
Which showed me a config file : /etc/init/jenkins
In here were the various settings I was looking for including the port so I made the change and started it up. In general I found the instructions for jenkins to be pretty simple and the interface to be quite self explanatory. I installed the android and git plugins via its plugin manager.  I also decided to install eclipse here along with GIT. I’m not going to be developing on my TV, but whilst the mac was undergoing longer term changes, I wanted to get a proof of concept up and running. I had a little confusion with the eclipse git plugin, since it seemed to expect each new project to be a new git repository. However you can make deal with lots of projects in a single git reop, as long as you effectively create a new project inside the git repo to start with, rather than in default workspace and then share. If the project exists inside a git repository then this is detected when you go to share the project and everything ties together nicely.

I decided I wanted a local repo, though much of the information on git assumes you’re going to use gitHUB to host your primary repo. I’m not sure what I’m going to do with the app yet, and I’m not comfortable making it public by default at this stage, and I’m not interested in paying for a private repo. Instead I initialized a git repo on top of my shared nas which is backed by 2 RAID-1 disks. So i figure with the primary supported on redundant discs and the nature of git being that I maintain local copies anywhere I chose to develop, it should be sufficiently protected.

So I have created a simple android project, and an android test project, which I have shared into a git repository. but how to get it to build? well it turns out google have even thought of this, I found some instructions that gave a command available in the ADT that inspects your eclipse project and creates an ant file to build it. Honestly this was shockingly easy.

Now that I had a git repo with some projects in, and a build.xml file that ant could use to compile it, I was able to configure jenkins with a build job that I configured to look at the git repo and poll every minute for changes. I added a prestep to aquire the android toolkit, and then an ant step pointing at my new build.xml. And that was it! I kicked a manual build and saw it picked everything up, compiled and created an apk file. I then made a couple of minor changes in my eclipse project and committed them, and watched as jenkins detected the changes and auto rebuilt the project.

So we’re in buinsess, but the iMac is not quite there yet, I got ubuntu installed, and installed eclipse, but a cursory check didn’t find the git plugin, and I fear it may not be available under the ppc version. In the meantime I used the wonder that is remote display export, and simple ssh -X to my asrock, and launch eclipse so that it displays on the iMac, it is perhaps not quite as fast as it might be running locally, but it does sidestep  entirely the issues of the ppc versus x86 architectures.

Next I need to figure out how to get the apk file published somewhere that my tablet and my phone can pick it up to install.

Its interesting to see how easy most of this stuff is now. Also it probably says something about me, that having had an idea of what I want to develop, before  I really do anything specific to that, what I wanted was source control and continuous integration in place. Next I will make sure that I have a test project running and able to run tests against the code. This is something I didn’t bother with when I wrote witter, and I really should have. The danger being that once I’d gotten a certain way without tests, it felt like it would be too much time spent not making progress on the app to go back and add them. Of course me being a software tester I really should know better, but  I figured it was just a coding exercise, I was just learning, it wasn’t ‘real’ development, until it was, and I was in a mess.

So this time I’m learning from those mistakes. I want the process for writing tests, building and running tests, and deploying to my devices to be worked out and slick before I even start to think about implementation of the app.

I’m sure I’ll post again as I learn more about the challenges of developing for android.

The first related presentation I went to was “Technology is your office”  By Horia Dragomir, The focus of this talk was the various tools and technologies used to help smooth team work despite never being in the same location. Horia himself has always worked from home, and currently works with a small team distributed over 4 continents. The first thing I noted about this example was that the entire team were remote from each other. So not a case where some people are co-located with others elsewhere, and I think that has an enormous impact. Where everyone is just as isolated unless they make the effort, I think everyone is trying to make those communication tools work, and ALL communication has to be made via these tools so its easy to keep up with what you want to know.

This was quite the contrast to the other talk I attended “Ain’t not cure for the distributed blues”  by Dan North aka @tastapod. Whose team is a small team located in London, away from the primary location of the company in Chicago . In this situation communication is happening in local teams, but the issue can be how to get communication working across sites, and remembering to do so. Despite the differences, there were some common recommendations.

Over share.

Common failings of remote teams amount to assuming that people know what’s going on, because it seems like such an obvious thing that as perhaps been talked about a lot around you, it’s hard to realise that people in other timezones have not heard the same things. Rather appropriately about this time I got an email from our architect expecting me to be heading to a meeting with him. I thought I’d made sure everyone knew I was out at QCon, I had announced it at a scrum, but I’d failed to really make sure it had been communicated.

Get together!

Yes, being distributed most of the time can work, but there is no substitute for getting together in the actual same place occasionally to form the kind of bonds of understanding you only get by spending social time with people. Dan North said that he had been to the Chicago office 4 times in the previous year, and one of those was for 11 weeks. This he felt was crucial for developing and maintaining the group dynamic required to be effective for the rest of the year. Horia Dragomir said that his team occasionally books a house somewhere so that the whole team can just get together, even though they work during the day just like they would normally, they then make sure the rest of the time is spent being social, going for walks, going to bars, whatever just to build up social experiences with the team. He even recommended conferences, if any two of your team want to go to the same conference, send them, their working relationship will only improve for the experience.

One thing that Dan said, that related to other comments I’d heard about metrics, was that if you think working distributed is bad, why? what measurements do you have to back that assertion up? If you can measure the team effectiveness such that you can show the distribution is harmful, perhaps you can come up with good ideas to mitigate that issue.  He also said only measure anything that you can get a trend for and know that a trend one way is bad and a trend the other way is good. Any other kind of metric is a waste of time.

The one interesting benefit Horia pointed out for allowing very distributed, at home working, was that you can get talent from where ever talent is. If you are in a major hub (silicon valley) then that’s fine there is lots of talent, but they have lots of choices and they tend to move around. If you are prepared to take talent and let them work from wherever they like, then you open yourself up to a much larger talent pool. I can certainly see the appeal of living where ever I like, and simply working from there, rather than commuting to a central location, and I can see how you could really make that flexibility work for you in terms of team productivity and commitment.

Dan’s whole presentation was called ‘There ain’t no cure for the distributed blues’ and I think it boiled down to some of the same things that Horia said.
Distributed is HARD
keeping it together is hard
Communication is paramount.

Mechanical Sympathy

This was the term used by one of the talks which was explicitly talking about this subject. The title of the talk was ‘Lock-Free Algorithms’ (Given by Martin Thompson aka: @mjpt777 & Mike Baker aka: @mikeb2701) but whilst the other talk ‘Scalable Internet Architectures’ (Given by Theo Schlossnagle aka:@postwait ) did not explicitly use this term, several of the things described hinted at the same basic idea.

That idea being that we, as software engineers, far to often totally ignore the nature of the real systems our software is running on, along our convenient high level abstractions to shield us from thinking about memory, or cpu cycles, or network packets. And in our complacency and general laziness we throw away all the hardware advances of the last 15 years by doing things which are horrifically inefficient.  Simply because we don’t stop to give it some though.

In the talk on lock-free algorithms, they showed how taking the time to construct a simple message queue between a single producer and consumer, which requires no locks could make things more efficient. But more than that, taking some time to consider the nature of the machine, the realities of exactly how a CPU handles memory, what bits inside a CPU are capable of what kinds of calculations, it is possible to make immense gains in performance (or perhaps avoid immense losses)
In their example they highlighted a couple of specific considerations. The first being that the default algorithm required a divide operation to calculate the position in the queue to use. Divide is a fairly expensive operation for processors. By making their queue length a power of 2, they could make use of the fact machines work in binary to construct a mask that allowed a much simpler operation to arrive at the same result.
Secondly, having given a whistle-stop tour of how CPU’s work, they talked about incidental sharing, where two variables, used together in their program would likely be allocated into memory next to each other. Each variable (both pointers) where relatively small structures, a 32bit int (4 bytes). However CPU’s access memory in 64Byte rows. this is the smallest unit they move, and when they access it, they effectively lock the use of everything in that row.
This means that on other core of your CPU can access any of the data stored in that row, and when it needs to it must not just wait for it to become free, but go fetch the latest version from where ever it was, potentially moving the data via slower memory in order to move it between cores or even processors. Obviously if the two variables are both used by the same thread that’s a good thing, but here, were they two variables are specifically used by the two parts of reading from or writing to the queue, this nature is very bad. By padding the size of each variable, they were able to ensure that they would be allocated into separate rows, and thus available for multiple cores to access simultaneously.

They measured their performance in terms of the number of million operations per second the algorithm could sustain. According to their numbers, the basic Java object for achieving a similar function is capable of 5Million ops per second, with latencies spiking at up to 32,000ns depending on the situation.
Their basic lock-free algorithm was similar to an equivalent lock free option in Java, capable of 15m ops per second, but with latencies down at a steady 180ns.
Then introducing their mechanical sympathy…they got up to 85m ops per second. Yes, I said 85 Million operations per second. So we’re not talking about ‘a little bit better’ we’re talking nearly 8 times faster.
Under the hood they showed that the number of operations sent to the cpu was broadly similar, about 4500 operations, however without the machine sympathy, that translated to about 63Billion cpu cycles, with the machine sympathies this came down to a little under 8Billion. This means the ratio of instructions to cycles was horrifically inefficient before, and all they did was make each clock cycle count more.

I’m a sucker for performance improvements, and this was a pretty dramatic example of how taking a little care can yield incredible results. (or being a bit lazy can cost you dearly)

This theme was picked up on in the scalable architectures talk, in particular in an example of considering the way data moves over networks. It doesn’t just move in streams of bytes, it moves in packets, and packets have a maximum size. He gave an example of checking the size of basic icons on a website, which might be incidentally just slightly larger than 1 packet. Ignoring this means you require 2 packets to deliver that icon. But perhaps you can squeeze it down a little, and HALF the cost to your network of serving that icon. We’re not talking about great leaps of engineering here, just caring enough to both doing a little maths, and understanding the impact of your decisions.

Theo discussed a number of factors that he felt the creators of software should know. That it is part of their job to know. It is too common to have software developers think the job is done when the software compiles into a binary. But that is just the very beginning of its life. We have an obligation to produce *operable* software that is actually fit for purpose, and a big part of that is understanding what its impact on the system it runs on will be.
When a customer hits a website, how much traffic will move over your network? how many calls to the back end will be made? How many database calls will that cause? what disk I/O? Are you monitoring all of these factors? Do you know what you system looks like when it’s healthy? If not how the hell do you hope to understand what went wrong when it fails? (and it will fail)

These two talks were both pretty different in terms of style, one being a very specific look at code examples and their implications, the other a high level list of things that you should understand about your architecture. But both left me with a powerful feeling that I don’t understand anything near enough about the details of the software we produce and I had better start finding out.

Throughout Theo’s presentation he provided us with rules, which I shan’t reproduce here except rule #10: Don’t be a F***ing Idiot (idiocy is bad)
What he meant by this was that it is really not that hard to run the numbers, to understand your system properly, and it’s not that hard to apply some logical thinking to the results of those calculations, in particular in questioning those which don’t sound right. Double check your working. Idiocy is contagious, if you stop caring to check things so will others and that is not a good culture to slip into.

I think these two presentations were my favourite of the 2 days of QCon I attended, and I thank the speakers for sharing their experience is such an accessible way. Now for the hard part, taking all my notes and trying to apply them to my own team and project. and try not to make angry with me.

If you are a professional, in ANY field, you should be going to relevant conferences. Seriously, look up the next one that is anywhere near by, and book a place on it. (or better yet get your employer to book you a place on it)

For years at IBM, I only went to conferences that were about IBM software, and I was only there to talk, I rarely had time to attend other sessions, and when I did it was constrained to the business of using IBM software, not writing it, so they were not really my field.
At my current employer, I have had the chance to go to a couple of conferences that are actually relevant to the field of being a software developer. Hearing ideas, techniques, war stories all related to what I’m actually doing day to day.

It is so easy, under the pressures of work to get yourself buried in a small world view. You don’t even notice it happening, its just work, you take what you know and apply it to solving the problems that need solving, and this may even go well. Whilst  your experience is your greatest asset, it can also narrow your view. When you have a hammer, every problem looks like a nail.

But attending a conference is a fantastic way to open you mind to new ideas, different ways of solving problems that may be better suited to the issues you face. Or even just have your eyes opened to the issues that are happening around you that you didn’t even notice, but really need attention paid to them.

I think that in my case it was a great injection of ideas that remind me how passionate I can be about the industry I work in, about the ways in which problems can be solved to make a real difference, and it makes me want to go learn more, try more, just do more than merely knock off the next task, raise the next bug, chase the next thing that is immediately in front of me.
Work is work, and some part of it will always be the thing you do to get paid, rather than the thing you wake up in the morning desperate to get started on. However I am fortunate in that my job does overlap heavily with something I love, and have always loved. writing software to solve interesting problems! Ever since I was typing in the games on the zx spectrum 48k! (love those rubber keys) from magazines that literally printed out the code listings (that seems like such a weird idea to me know, but at the time it seemed great).

I *like* taking these multipurpose machines and making them do my bidding, I *like* optimizing something to go faster and be more efficient. I *like* taking something manual and making it automated. And it turns out there are a whole load of tools, techniques and languages out there that I’ve not played with yet that allow you to do all of these things in new and interesting ways.

I can only imaging it is the same for all fields, learning from those more experienced, or even just those with different perspectives, is just downright invigorating and can surely only lead to you being better at what you do. Learning really shouldn’t stop once you walk into your first job, qualification in hand.
Taking a course might be a good for some people, but that really implies you already know that you need or want to know more about a particular subject.

A conference is a great way to sample a whole load of ideas and subjects in a short space of time, like an ideas buffet, and actually just like at a buffet, I found my biggest problem was simply not having room to try everything on offer, being forced to chose between some great subjects on in the same time slot was at times torturous. To some extent twitter helped, as you could pick up some highlights from other talks after the fact. Though on one occasion I chose badly and wound up in a panel discussion that really did nothing for me, and spent the time jealously observing the tweets of others who had chosen better.

Whilst at the conference I took a lot of notes, really a LOT of notes. so I will likely write up some thoughts about a few of the specific sessions I attended in future posts. In making these notes, I discovered that my transformer prime was practically *made* for this kind of usage. I literally spent 6 hours (6 sessions) a day, making notes, reading tweets, checking mail etc etc. all without every carrying a charger with me, and still having enough charge to sit on my sofa in the evening browsing on the tablet. Whilst I drained the keyboard battery every day, I was never unable to use it to type during the sessions, and the tablet never dipped below 50% charge even after a day of practically continuous usage. Given it’s also super light, it was really easy to carry around, sit on my lap and be productive with all day without a problem, and I didn’t have to play hunt the plug socket at all. These things really come into their own in this environment.

My one complaint to the qcon organisers (sponsors?) relates to the end of each day. There were beers provided for general socialising and winding down after a day of absorbing information. But it was just literally that. beer, no other choices. I am perhaps unusual in that I really don’t like beer/larger. So the absence of another choice (wine? cider?) felt like an omission. So if qcon is reading this, see what you can do about that for next year?

It was an hour into my first experience with designing with openSCAD, when disaster struck. But I’m jumping ahead, lets go back to the beginning…

If you’ve looked around my blog you’ll know that I have a desire to make a CNC router. I have constructed what might pass for the 3-axis mechanism. However, I lack any of the electronics, motors etc required to make it go. There is not an insubstantial cost to getting from where I am now, to a working machine. In the last year I have not really been able to justify taking things further, but it does remain a firm plan. One day, I will have a cnc router.

Part of the difficulty justifying the cost, is that difficult question ‘so what are you actually going to do with it?’ to which ‘I’m gonna make!…erm, stuff!, and the computer will control it!’ is not quite good enough. So I need ideas that can be fulfilled with a cnc router. On top of that, I need to know how to take those ideas and get them to the point that the only thing missing is a cnc router.

Enter Computer Aided Design (CAD), cnc machines take instructions in the form of tool path vectors, this information needs to come from an accurate computer model of what you want, and that all starts with CAD. There is no point having a cnc anything if I’m still drawing on scraps of paper and totally out of scale.

And so my journey began, I need to learn to compose my ideas in CAD software, so that one day I can process those files to my cnc machine. And until that day I can use the designs even for hand construction, even if it is slower and less accurate. So job one, find some CAD software and try to mode something. In this case I’ve been kicking around an idea for a rotating wine rack, something that will fit in a small under-stairs cupboard and maximise use of the space. The base of this idea is an octagon platform with supports to hold bottles in layers that interlock slightly.

So I want to draw an octagon in some CAD software….how hard can it be? I fired up Autocad, and set to it. cubes, right no problem, but how do I make an octagon? or a polygon generally? I found the interface very hard to use, working on the 3d space trying to figure out which clicks, where would activate various features. After some considerable messing around and getting no where, I decided to switch to Blender to see if I had better luck. But again I found myself frustrated by an inability to accurately control the interface. Perhaps with training and time I’d appreciate how fast and beautiful it is but for now I was getting no where fast and getting annoyed.

Then I hit upon openSCAD, this is a completely different approach, in openSCAD you write script to describe what you want, you get a bunch of standard primitives, a way to define polygons to extrude, and a set of translation, rotation, union and difference commands with which to maneuver and compose objects. Now, programming is what I do all day long for a job, so this felt like home to me, picking up the language was pretty easy and before you know it I had my octagon base and the first draft of a set of wine bottle supports

Then disaster…yes, openSCAD crashed. I think I may have accidentally told it to compile/render whilst it was already doing so, whatever happened it crashed. There was a chance to attach a debugger which I hoped might help, but it didn’t work and the app went down hard.

I fired it back up to discover that openSCAD has no auto-save feature, and furthermore hitting ctrl+s *doesn’t save* because some idiot made it F2…!”$”£%£”$.

I like most in my profession hit ctrl+s pretty much on autopilot every few seconds, anytime I pause to think, any time it feels like it’s been too long. lots of hard lessons about saving are ingrained in a compulsion to save constantly. But all of that was for nothing, as hitting ctrl+s wasn’t doing anything. And so I lost about an hours work and had to put the laptop down for a while before I could even consider starting again. This was not a good start to my experience with openSCAD.

However, it really is much easier for me to use than the more ‘traditional’ CAD software, I understand it much more, and having gotten the hang of it, it took relatively little time to get back to where I was and beyond.

One thing I really like is that you can make anything into a module, then call that multiple times. you can include files as libraries, so it was easy for me to have 1 file in which I define the ‘parts’ of a design, then another in which I call them and move them into position to render what the finished item is supposed to be. Then finally a third file which calls the parts and just renders them into a form I can use as a template.

I can figure out how best to layout the pieces to us the least wood possible. When happy I just aligned the view to front, and printed out the image. It took a couple of guesses to get it to print at 1:1 scale, so that I could literally cut the pieces out and use them as templates. But this worked pretty well and made things easy in the garage when I marked everything up and cut things out.

Obviously this is also required for my cnc router, whilst it’s nice to have a file which renders what the finished item is going to be like, what the machine will need to know is what shapes to cut out of a flat piece of wood.

In all I consider this to have been a success, though you can judge for yourselves when I blog about the finished wine rack. I will certainly use openSCAD again for designing things, and getting everything to scale and figured out before I start cutting wood. I will learn to hit F2 on a regular basis, which is actually helped by having the multiple file setup, since you need to save the parts adjustments before you can re-render the views. And one day, I will have a cnc router to do the cutting.

So more recently I updated that program to split things into separate processes, instead of separate threads. This way I got better isolation between the main runner and the tests being run, and I got to work with tests that each wanted to mess with the spring container, as they are each in their own process so they don’t interfere.

Now the biggest challenge to running most of our ‘local in memory’ tests was that they were using a real sql server (somewhat heavy weight for what are technically ‘unit tests’ ) However a colleague recently altered those tests to use an in memory database, such that each new test fixture was playing in it’s own space, so it was no longer an issue to run multiple tests that attempted to update the same things at the same time.

Now this test bucket had been growing steadily in size and runtime, at the point that it took about 45 minutes to run all the tests in the bucket, we split it into 3 separate buckets based on projects, so that we could run them on 3 build agents. This obviously meant that we were using 3 times the machines, to keep the runtime down to an acceptable level. Again the runtime crept up until each of those 3 buckets was taking about 25 minutes. It is at this point that I started experimenting applying my multi process runner to these tests. until now I’d only applied it to the selenium tests written by my test team, but I’d kept the tool generically nunit, so it was time to play

Our standard build server is a 4 cpu win 2008 machine, with about 2gb of memory allocated, (these are all virtual) so I took the whole run, and used my parallel runner using 4 processes (one per core) and got the whole bucket (~1800 tests) to run in about 25 minutes. So it was no faster than the total elapsed time we already had, but using 1 build engine instead of 3. This was a pretty good result, but not good enough for me…

One issue I had was that the developer written tests are spread over a bunch of dlls, and the nunit framework only lets you give a single runner a single dll. The way I originally wrote the code was based on a single dll with all the tests in, and when I enabled it to work with lots, I basically just wrapped the whole logic in a giant foreach loop. This was pretty inefficient  for dlls with relatively few tests in. So I refactored it to pre-load all the dll information, so that it would keep allocating work to free processes as long as there remain more dlls with tests in, rather than wait for a single dll to finish all its tests before starting on the next.  making this change pulled down the runtime to around 22-23 minutes, a couple of minutes of runtime just for handling the process allocation logic a little more sensibly. (I know it doesn’t sound like much, but we are talking around 10% improvement here!)

At this point I asked our infrastructure team to hook me up with a build engine that has 8 cpus assigned, instead of the standard 4. This allowed me to boost the number of processes to 8, in theory double the cpu, double the speed, so I had high hopes. I was pretty happy with the results, the change brought the runtime down to about 13.5 minutes! now we’re talking. It’s not a liner improvement, but suddenly things are looking pretty sweet, we can use 1 build engine with the cpu allocation equivalent of 2 of the old build engines, but achieve half the runtime for the entire suite of tests compared to using up 3.

Now, drunk on the possibilities of speed, I scoured the code to figure out ways to squeeze a little more performance out. In doing this I spotted some logic I had put in place for the selenium tests, after launching a new process i actually sleep for 5 seconds to give it some time, this is because I found the selenium server would fail to create its temp folders if multiple processes talk to it in very quick succession (seems like a bug to me) but this sleep was totally unnecessary for the in memory unit tests, so I made it configurable, the existing tests could keep the sleep, but the new ones would reduce it to zero. I wasn’t sure how much impact this would really have, since after the initial launch of processes we could be waiting in other places.
However having made the change I wiped about another minute of the run time, now getting an average of 12.5 minutes per run. Again, 1 minute off may not sound like a big deal, but if you’ve read this far in a post about making tests go fast, I figure you are the kind of person that knows how much of a difference it can make to shave those extra seconds off of a test run. At this point we’re chasing the golden possibility of having a serious commit test bucket, running 1800 tests, taking less than 10 minutes. (not quite yet, but I’m so close I can taste it)

Unfortunately at this point I’m also out of tricks, I’ve looked the code over and there are no obvious places to make a difference. One great hope was a change made by one of the developers to persist a version of the in memory db and deserialise it into memory rather than recreate it from scratch each time. This seems like a great idea, and made a big impact when running the tests serially, but in practice it yielded no appreciable change to running them in parallel.

So I was chatting with the dev architect, and told him that this is as good as it gets, there are no more software improvements I can think of, the only thing left is throwing more brute power at it. So he did…

I came in the next morning to find an email showing the cpu load on a build agent with 24 cores! and 8gb ram!, the runner was pushing it pretty hard, and achieved a runtime of just 5min30seconds, it has to be said I was pretty damn happy with that result. Sure it’s not a linear improvement, but holy cow, we took a test run that took over an hour run serially on one build engine, and dropped it to barely over 5 minutes, the product takes longer than that to compile.

It’s possible that we won’t leave a build engine with that much cpu assigned all the time, however it is great to know that when we want to, we can turn the dial up to 11 and really help crank through the changes.

There are many professionally written reviews all over the web, and all found it to be the best android tablet on the market right now. There has also been a fair amount of press around reported short comings and failures.
When reading xda-developers prior after making my pre-order and before it arrived, I was beginning to get a little nervous with reports of dead pixels, DOA units, units without serial numbers preventing updates, wifi being hopeless… seemingly many people were unhappy.
However, the thing I clung to was that all those professional reviews couldn’t have messed up, if the thing had real serious issues it would never have had such glowing write up.

When my unit finally arrived, I waited patiently to play. I was at work when the package arrived, and knew that if i opened the box to play I would get nothing done. So I waited.

Eventually the evening came and I opened up the box and plugged in. I did immediately try to power up, but got no response. So I left it to charge. In total it took about 50 minutes on charge before there was enough juice to respond to attempts to power up. I realised that most (all?) claims of ‘DOA’ were likely people freaking out that it didn’t respond once plugged in, not realising the amount of time that might be required on charge before it would respond.

Of course the first thing I checked for was dead pixels, and was relieved to find one. Then I checked serial number and that was fine to. I let it download all my apps (I have an android phone so it synced all the apps installed there to my tablet)
Once that had finished I went checking for the Android 4 update (Ice Cream Sandwich) and found it waiting, so pretty much immediately I hit the upgrade path and picked up the new OS level, and a camera driver update.

After the update and reboot, I was playing wth ICS to see what was what, when I realised that the wifi did keep dropping out on me, and re-establishing. Perhaps this issue was a real one, and at the rate it was happening it would have been a pretty serious issue. But then it occurred to me that I’d had no such problems pre-upgrade, and my instincts said to try a clean power down and back up. (my phone sometimes has the GPS go mental, tracking my position about 100 yards offset from my position, a reboot always clears it up) As suspected, a reboot cleared up the problem, and a week later I can say I’ve had no issues with wifi dropping. Some say the range is poor, I can’t say I’ve noticed, I get a good stable connection everywhere at home and at work, so I guess it’s good enough.

After a week of paying I can honestly say I love it, all the awesomeness of a tablet, but with the addition of a keyboard (on which I’m typing this) which is a little small, but pretty good for writing long form content without messing around with the on screen keyboard. Also the battery life is at least as good as claimed. I have to say I’ve not pushed it that hard, I charge it every night, since that is the habit I’m in with my phone. However today, I sat in bed this morning reading engadget’s distro magazine, my wife and I watched the 2 hours of ‘star wars uncut’ (pretty cool, check it out) and now I’m writing this. The tablet shows 82% charge and the keyboard is at 52%. if I tried this on any laptop I’ve ever owned it would be on its last legs by now. And my phone would likely be looking in serious need of a top up after that much video played, and streamed over wifi.

One complaint that all the reviews made, was not specific to this tablet, but rather the continuing issue with all android tablets is ‘not enough tablet apps’.
I can now appreciate what they’re saying, but I think it’s not what you might think. Every android app runs, and just expands to fill the screen. For many that is fine, I’m using noteeverything right now, and it works great at tablet size. However when you use those apps that are ‘tablet optimised’ you do feel like you get such a nice experience, that it is a little disappointing using apps that don’t make the most of the large area and resolution. So it’s not so much that you can’t get apps to do stuff, just that some of them leave you wanting more from the interface once you’ve seen how good it can be. I guess as those reviewers noted, it’s just a case of time.

Obviously you can’t really give a fair review of any device like this based on just one week. time will tell. However so far I love it, have had really none of the

reported issued, and find it to be every bit the amazing device that the early reviews reported. If you are on the fence, get off it and go buy one, you won’t regret it.

I wrote this post entirely using the wordpress for android app on my phone, which then decided to lose everything I’d written ;-( So here I wrote again thinking that I may never trust the android app again.

Previously I wrote about having made an oak book stand for my wife’s Christmas present. In addition to that I also decided to make her an artists sketch pencil.

We originally saw these at a craft faire in Winchester, as part of a large selection of pens being sold by a woodturner. Kat and I were both drawn to the unusual nature of these sketch pencils. Basically the mechanism is a 4 jaw ‘chuck’ which you operate by pushing the end plunger to open up the jaws. They take a 5mm lead in various colours, and when released, the 4 jaws grip the lead securely.

The guy selling them mentioned that you can’t get the kits in this country and they have to be imported. I forget now much he sold them for, but they were not cheap. I often feel a little pang of guilt at this kind of time, since I’ve normally shown quite an interest in the goods being sold by a woodturner, but only ever out of interest to go away and make them myself, never to actually buy them.

Having remembered them I did a little search and found them sold by Craft Supplies USA Woodturners catalog, unfortunately having to get them from the US makes things a little costly due to the shipping, so I ended up buying a couple of those sketch pencils and a couple of their ‘tool box’ variants. The tool box variant have little hex ‘nuts’ at the centre and plunger end in order to stop the pencil from rolling around when put down.

In construction I had a couple of mishaps from not really reading the instructions on drilling the blank (how hard can it be to drill a hole through piece of wood?) Basically I had the drill speed way to slow, and didn’t leave enough spare length on the blank to allow me to stop without drilling all the way through. So on two separate blanks I managed to blow out the base of the blank as I got to the end of the drilling (the bottoms literally exploded into fragements ;-()  This left me not very happy. However after taking a little more time to read up on it, I sped up the drill, made myself a better jig for supporting the blanks, and went more carefully and all was well.

The tool box variant was a little bit strange, in that the rear hex nut is not a push fit into the barrel, rather having turned the barrel you need to trip a few mill off the rear end, back down to the brass tube, then glue the nut onto the brass tube. This also means that there is no snug fitting around the plunger (as there is on the artists variant) The finished product is still cool, but I feel it would be nicer with a push fitting that provides something close to the rear fitting of the artists sketch pencil.

I think this made a really nice gift and I was happy with how it turned out. A colleague at work expressed an interest in them, but I think the cost shipping would make them quite difficult to make and sell at a reasonable price. That said I do also have my eye on the soft touch kit for making stylus’ that work with capacitive touch screen tablets (my transformer prime should arrive tomorrow) so maybe I could order another batch of kits. If I convince people to buy them from me I might offset the cost of making my own…