Revolution Systems Blog

ResumeBucket.com Launches

I hope everyone had a great holiday this year.  For the past few months I've been working on an online resume site ResumeBucket.com and I need your help taking it for a test drive.  Our goal with this site is to create a site where you can upload your current resume in Word form, build a new resume using our online resume creation tool, or even just type in what you want using using our online text editor. 

The site gives you a unique URL you can give out to friends and prospective employers so they can instantly access an up to date copy of your resume.  There are options for them to also download a copy of your resume in Word or PDF format.  For an example of how your resume can look see our CEO's resume page.

Unlike the other resume services out there, employers are able to search the database without paying any huge fees which will drive more qualified employment leads to your INBOX.

Please take a few minutes to kick the tires.  You can leave feedback here in the comments or E-mail me at frank@revsys.com. Thanks!

Why isn't PostgreSQL using my index?

This is a common question I see from PostgreSQL users.  In fact, someone was just in IRC asking it and it prompted this post.  The exchange usually goes:

Steve: I have this column foo and I have an index on it, but when I do SELECT * FROM table WHERE foo = X, EXPLAIN doesn't use the index.  What am I doing wrong?

90% of the time the answer is unfortunately "Because the query optimizer is smarter than you are". Or maybe it's fortunately depending on how you think about it!

In this user's particular case he was mocking up a database schema and had only one row in the table he was querying against.  People who are more familiar with PostgreSQL will probably roll their eyes at the question, but if you put yourself in the user's shoes I can see how people would be confused by this.  They are thinking, "I put an index on there on purpose, why the hell isn't it working?". 

PostgreSQL's query optimizer is smart, really smart and unless you have evidence otherwise you should trust what it is doing.  In this particular case, the optimizer realizes that if a table has only a few rows that using the index is actually slower than just spinning through the entire table.  Just because PostgreSQL isn't using your index today with a small number of rows, does not mean it won't choose to use it later when you have more data or the query changes. Because he was just mocking up a design he didn't have real world data, which is almost always a bad way to performance tune your system unless you are very familiar with how PostgreSQL behaves. 

Now there are other reasons why it might not be using the index.  If you have lots of data and the query you're running appears that it would benefit from the index, it might be a simple matter of forgetting to run an ANALYZE on the table or not having autovacuum turned on.  Until PostgreSQL re-analyzes your table it doesn't really "know" about that index to take it into account when building the query plan. 

While performance tuning PostgreSQL is much easier and better documented than in days gone by, it can still be very confusing and time consuming for the inexperienced.  If your business needs help tuning their system you might consider my PostgreSQL Tuning Service.

Fret Free -- Introduction to Django and the Django Software Foundation

LinuxPro Magazine just released my latest article, an introduction to Django and some discussion about the newly created Django Software Foundation. Being a life long Perl user, I didn't think I would enjoy Django at all. I have to admit that it is a VERY polished system.  It has great PostgreSQL support, in fact the core developers smartly prefer it over MySQL for their own systems.

You can download a PDF copy of the article at, Fret Free -- Django and the Django Software Foundation.  The print issue will hit the stands in October.  Hope you enjoy it!

Installing Apache2::Request on a 64-bit system

I usually shy away from using 64-bit systems unless there is a clear need for it.  But with more and more hosting companies installing 64-but Linux distrobutions by default I sometimes have to suffer through.

Today I ran into a very small bug in installing Apache2::Request modules via the CPAN shell.  This simply installs the default libapreq2 library with the Perl bindings for you. When attempting to start Apache I received this error:

Can't load '/usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/auto/APR/Request/Request.so' for module APR::Request: libapreq2.so.2: cannot open shared object file: No such file or directory at /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/DynaLoader.pm line 230. at /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/APR/Request/Param.pm line 27 Compilation failed in require at /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/APR/Request/Param.pm line 27. BEGIN failed--compilation aborted at /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/APR/Request/Param.pm line 27. Compilation failed in require at /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/Apache2/Request.pm line 2. BEGIN failed--compilation aborted at /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/Apache2/Request.pm line 2.

Basically it is saying it can't load libapreq2.so.2.  Digging around in the system I found it had installed them in /usr/lib instead of the proper /usr/lib64.  The actual shared library was compiled for 64-bit, but was just installed in the wrong location.  A quick symlink like this fixes the isuse:

ln -s /usr/lib/libapreq2.so.2 /usr/lib64/libapreq2.so.2

Hope this helps you get over this small issue.

Django Software Foundation

The Django guys have scored another milestone today with the creation of the Django Software Foundation. Being a huge Open Source advocate I love hearing great news like this.  You can read more about the foundation at:

• Announcing the Django Software Foundation
• New Foundation for Django

On an unrelated note, an article I wrote for LinuxPro Magazine last November about using Perlbal was recently put online.  You can read it at The Juggler -- Let the nimble Perlbal webserver keep your traffic in balance .

Test Driven Development and Getting Things Done

I'm sure someone, somewhere, has already determined this.  It's probably been blogged about before, but I just realized something today.  Test driven development dovetails nicely with David Allen's time management book Getting Things Done.  By creating your tests first, with or without even marking them as TODO tests, builds you a TODO list of sorts for your project.

Other than just being a different way of doing things, I never really took to test driven development for small or personal projects.  Most of my projects are web applications and "unit testing" modules never seemed worthwhile because you weren't easily able to test the database and web interactions.  Well I can easily say that Perl Testing: A Developer's Notebook by Ian Langworth and chromatic has shown me the way.

I had always focused too much on ensuring that 'make test' would be able to be run from anywhere and simply did not think about the fact that I could automate most of the work of building a test environment and if I wasn't able, to bail on those tests so they would be skipped.  For example, you could automate creating your PostgreSQL database, but what if PostgreSQL isn't installed on the server or what about when your user isn't allowed to create databases?  You just bail out of those tests.

I was definitely too concerned with getting 100% of the tests to pass 100% of the time, in all possible scenarios now that I think about it. 

Want to be a better manager?

I've been reading a great book recently titled First, Break All the Rules: What the World's Greatest Managers Do Differently. I highly recommend it for anyone who manages employees, but it makes two great points early on that are especially appropriate for technology managers:

Treat your Employees Differently

You should treat your employees differently.  Each has unique strengths, weaknesses, and differ in the way they learn and you should capitalize on that, not try to homogenize everyone. Sally might be an awesome systems architect, so you don't want to weigh her down pounding out mundane features.  Bob might be the fastest developer and giving him those same mundane features makes sense.  Steve on the other hand might be your more efficient debugger, etc. 

They learn in different ways, some by reading, others by doing, and some need a more traditional classroom environment. Sending a do-er to a lecture class is a waste of everyone's time. 

Don't Let Human Resources Get in the Way of Your Hiring

I've touched on this topic before, but you can't leave the resume collection and filtering process to your HR department.  Even in our more tech savvy age, I still hear horror stories from programmers how they lost out on being interviewed for a job because the HR person didn't know J2EE or J2ME were 'Java', that Ubuntu is a Linux distribution, or that having 6 years of mod_perl experience probably means they know Perl pretty well.

A great technology manager has to hire a person based on several things, one of which is experience with a particular technology or technologies.  But that's only part of it.  Troubleshooting skills, breadth of experience in many technologies, personality and how this new hire will fit into the overall team structure are just as important.  The worst possible case is when the manager is barely or not at all involved in the hiring process.

Now there are exceptions to this.  The best example is my friend Josh Stomel who runs NeoHire. Josh is one of those rare recruiters that actually understands technology and more importantly knows when he doesn't know and has a huge network of geeks like me to ask questions. You can just hand NeoHire a list of qualifications and they will bring you back a group of top notch talent.

So it is possible to have HR deeply involved in the hiring process, it just has to be the right people. 

Building mongrel on Solaris

I keep running into this same situation, so I figured I would write a post for everyone's benefit.  When trying to build recent mongrels or some other gem dependencies such as fastthread on a Solaris system the automatically generated Makefile is all sorts of messed up. 

This testing was done on two different Joyent Accelerators, your milelage may vary on other Solaris installations.

The steps are pretty easy:
1. Attempt to install the gem via the normal 'gem install mongrel'
2. This will fail, but will download the gem for you.
3. Move into your gems directory.  On these sytems that is /opt/csw/lib/ruby/gems/1.8/gems/
4. Move into the particular gem you are building that is causing you trouble. In this case mongrel-1.1.3
5. Move into the ext/ directory and the subdirectory that can't be built.  You'll see it in the error messages from step 1.
6. Edit the generated Makefile as such:

    Make sure CC is pointing to your C compiler, I had to edit this to reference gcc rather than cc.
    Make sure LD is pointing to a real ld tool.  On this system that meant I changed ld -G to gld -G
    Edit CFLAGS to be CFLAGS= -03 -I/path/to/libs Essentially you just need to remove the -KPIC and all     -x es from the generated file.

7. Run make install
8. Copy the gemspec file to /opt/csw/lib/ruby/gems/1.8/specifications so it will be registered with the gem tools.

Hope this helps.

PostgreSQL version 8.3 Released

I just got word that version 8.3 of PostgreSQL has been released.  Along with the usual amount of improvements there are some new features in 8.3 that should be of interest to PostgreSQL admins and developers such as:

• Integrated TSearch
• ENUM and UUID data types
• Faster sorting technique used for LIMIT operations
• Faster LIKE and ILIKE operations
• Lazy XID assignment which will make many read only operations much faster

Check out the full list of features at the PostgreSQL site or download it from the download section of their site.

EveryBlock.com is now launched

My friend and former co-worker Adrian Holovaty and his team just launched their new project EveryBlock.com. EveryBlock takes the term hyperlocal to a whole new level.  They aggregate tons of public data sources by geo location so you can for example find all of the recent crime around a particular address, neighborhood, zip code, etc.  Or maybe you might be interested in the building code violations of where you live or work?

Right now they have San Francisco, Chicago, and New York up and running, but will be adding more cities as time goes on.  Adrian asked me to help performance tune their PostgreSQL database a couple of months ago and so far things seem to be humming along nicely.

Here are some links to other blogs talking about EveryBlock.com:

• News Kids on EveryBlock 
• Launching EveryBlock.com
• EveryBlock, an especially slick interface for discovering local information
• A Review of EveryBlock: How it helps the public

Congrats and the best of luck to EveryBlock! I'm sure we'll see even more new and interesting things from this team in the future!