State of the Union

Quick update on where things stand after a hectic few months of infrequent development.

Solving Two Problems
My Thesis advisor jumped in and helped with my Two Problems. The answer to the MySQL install was to fiddle with it until I lost my mind. Then to follow my advisors advice and remove the second copy of MySQL that was running from the Bitnami Rails Stack. I bailed on Bitnami after the first few weeks of dev, but I never bothered to remove it until it burned me.

His solution to the logging problem was priceless. I said "It hurts when I try to do logging this way." He said "stop doing logging that way." [Hand smacks head... DUH!] I was so fixated on getting it working I forgot to check if it was a good idea. Linking the data import script to the webapp logs was not only painful to implement, but would have been painful to use. So i just set up a log file for the data importing process. QED.

Development Mechanics

I develop data migration code in script/import/archive.rb. I start by running that code...

     ruby script/import/archive.rb

...and fixing any bugs. This allows me to develop the data migration code and test it without spinning up a new rails environment every time I want to try something out, but it also means I cant do end to end testing legacy-data to new database data. Once archive.rb is working I move that code into lib/thesis_importer.rb and test to make sure the data is actually making it into the DB correctly.

      script/runner script/import/project.rb

 It's dirty scripting/data-manipulation work, but it's gotta be done.


Currently Working
Everything from site_properties.xml is now importing correctly (except for the bug_list_column_order). That includes a bunch of data transformations from the old file based schema to the new relational schema. Now when I run project.rb it

1. creates a new project in the data base
2. populates all the immediate data like name, projections etc, 
3. links all the constant tables like bug_too_url, 
4. finds or creates a person object for that administrator and links them through the project_administrator table.
   
5. writes out a useful log file about what it did
6. doesn't produce any errors

The problem with bug_list_column_order is that I made a mistake with the original schema. The schema is missing a way to set the default bug column display order for a project. I have it on a per bug basis in the bug_lists table , but not for the project. In the legacy system it used to be set for the project and overridable

by each bug list.

So I need to:

1. add a t.string :table_column_order to the project table (still considering comma separated string of column ids to handle order. not convinced this is a good idea yet.)
2. Update the YML file for the fixture for bug table columns to have the column data from the legacy constants file
   
3. write a mapping function between site properties and the new settings 
4. put the default order into the Projects table during import.

Stuff I did today

1. Cleaned up a bunch of strange stale bugs in thesis_importer.rb that were keeping it from running.
   
2. Got site administrators populating correctly
3. Wet sledding with my kids.
   
4. Got ButToolUrl constants in the database and importing from site properties working correctly. 
5. Fought with a CPU maxing out issue in Aptana Studio. Googled it silly and still didn't find a solution so I logged a bug. I wish I was doing this on a mac...
6. Wrote this post.
   

42 days left till the due date. It's crunch time.