Buildbot

Buildbot is a continuous integration tool that runs automatic tests at every code update.

The OpenERP buildbot configurator integrates the OpenERP buildout recipe, so it allows to automatically test OpenERP instances based on buildout. The instances can be generic or personalized for customers.

The main benefit of using Buildbot instead of runbot is that buildbot allows to test a specific configuration (a specific revision of the code, possibly with patch, a specific set of modules) and does not force to test all available modules (like runbot)

Localizations Tests

Our case: buildbot for Italian and Swiss localizations.

On http://buildbot.openerp.agilebg.com you can find the AgileBG public instance of buildbot. Clicking on Builders you will see all the configured OpenERP instances with the result of their last build.

For example in the build 7, composed of OpenERP 7 and Italian modules, we can see several sections.

‘Steps and Logfiles’ contains all the steps made by the build. Starting from the code update to the database creation and the tests execution.

Step 13, the only one red, is the step that checks the tests result. To understand the meaning of the failure, we need to open the output of step 12. Here we can find 3 errors:

1. ValueError: res_partner_c2c not found when processing /home/buildslave/public/openerp/openerp-italia-postgresql-9_1/build/parts/italian-addons-7.0/l10n_it_fiscalcode/test/fiscalcode.yml. This Yaml file appears to depend on missing data. This often happens for tests that belong to a module's test suite and depend on each other.
2. File "/home/buildslave/public/openerp/openerp-italia-postgresql-9_1/build/parts/italian-addons-7.0/account_invoice_sequential_dates/invoice.py", line 44, in action_number _('Cannot create invoice! Post the invoice with a greater date')) except_orm: (u'Date Inconsistency', u'Cannot create invoice! Post the invoice with a greater date')
3. File "/home/buildslave/public/openerp/openerp-italia-postgresql-9_1/build/parts/server-7.0/openerp/tools/yaml_import.py", line 501, in _eval_field raise KeyError("Object '%s' does not contain field '%s'" % (model, field_name)) KeyError: "Object '' does not contain field 'address_contact_id'"

About the error 1, I opened the 1295071 bug. Releasing the relative patch, buildbot noticed  the change and ran the build 15 (All Changes section). That specific error doesn’t appear anymore, while the other 2 are still present (about error 3: bug 1295743).

In order to get a green build we have to wait for every problem to be fixed.

But the Swiss localization modules are green: for example build 7 of the openerp-swiss-7-ocb-postgresql-9.1 instance. Here we can see in step 13: “No failure detected”.

Specific Configurations

AgileBG uses buildout + buildbot for every customer’s instance: keeping green tests guarantees the best stability for every system.

Thanks to buildbot’s way to test specific configuration, having a red openerp-italia-7-ocb-postgresql-9.1 instance doesn’t mean that customers that use the Italian branch have red tests: they would have red tests only if their configuration included the modules whose tests are failing.

Update october 2014

After the transfer to GitHub, the tests are now automatically executed by travis. Every OCA project has its own test environment. Taking for instance the Italian project, you can find the branches status at https://travis-ci.org/OCA/l10n-italy/branches