5 tips for better scripts
From generating UUIDs for a new column to fixing legacy data, we need to write a custom script to achieve this.
In this post, we will see together five tips that will help to improve the quality of those migration scripts.
Tip 1: Get to know your data
It’s easy to rely on assumptions and we often expect that the latest code that we are reading reflects what we have on the database. For example, we can open a model file and we can see the following validations:
class User < ActiveRecord::Model
Validates :age, presence: true
Validates :email, uniqueness: true
# more code
end
And we can check the schema file where the email has a unique index on that column.
create_table 'users', force: :cascade do |t|
t.string 'email'
t.integer 'age'
t.index ['email'], name: 'index_users_on_email', unique: true
end
But what if those validations were deployed only a couple of months ago and the index for uniqueness was added after reading this post.
So if we have a script that requires the presence of age and email values. We have to do some queries and answer those questions for exp:
- How many users don’t have an age value?
- Do we have duplicate email addresses?
Those will be edge cases and it may need to be addressed case by case.
As part of knowing the data, It’s a good practice to check the number of records that will be affected, it doesn’t only help us to estimate how long it will take to process the script but it will be useful to share it as part of the report.
Tip 2: Don’t let the script explode!
Imagine you run a script and there was an issue with a record, unfortunately, the script will be interrupted and you may need to handle this edge case. So what can we do to prevent this?
There is no magic, a migration script is still a code and we can use error handling. For example, we can catch the generic error and store the records ids as reference.
Here is a snippet that uses a simple array to store the ids of problematic records.
records_to_check = []
records.each do |record|
begin
# some code
rescue e
records_to_check << record.id
end
end
In some cases, it may be better to create a log file but it still follows the same approach :wink:
Tip 3: Show the progress
How many times, did you ask “is this script still in progress?” No need to be fancy here, displaying a simple dot after handling a recording is more than enough. Use print statements to check that your script is still running, don’t use print with new line return :smiley:
Tip 4: Ask for reviews
We introduce code changes through Pull requests, so what about leveraging this system for our scripts? If the script is part of some implementation, I would copy it in the description of the PR and ask my colleagues to review it.
There is also another approach, that I have seen used by some teams. The approach consists of using the migration option in frameworks like Rails. But keep in mind, you have to define rollbacks like this example:
class CustomMigration < ActiveRecord::Migration[6.0]
def up
# migration code
end
def down
# rollback logic
end
end
Tip 5: Let’s test it!
We care about quality so we need some testing backgrounds, development and, staging environments fit this role.
In some cases, it’s nice to do a “Dry Run” on production. Just comment all execution parts (updates, changes…) and use print output instead to see how it goes.
This is our safety net and don’t be surprised if you managed to catch an edge case here :wink:
Finally To production!
Now, let the script goes to production, but we keep monitoring the system (dashboards, error analyzers,…) and we don’t celebrate until everything passes successfully! Please don’t be that developer who runs the script and goes for a lunch break.
Conclusion
We saw together five tips to write migration scripts. I would consider those tips as just the foundation. Certain cases may require to consider other aspects, like performance and optimization.
That’s it, stay safe and happy coding :slightly_smiling_face:
