samedi 3 janvier 2015

Non-deterministic behaviour in Node.js


I have a Node.js application that, upon initialisation, reads two tables from an SQL database and reconstructs their relationship in memory. They're used as constants throughout execution.


Problem: Sometimes I can't access the data, even though the application reports successfully loading it.


Code:


constants.js



module.exports = {
ready: function () { return false; }
};

var log = sysLog('core', 'constants')
, Geo = require('../models/geo.js');

var _ready = false
, _countries = []
, _carriers = [];

function reload() {
_ready = false;

var index = Object.create(null);

return Geo.Country.find().map(function (country) {
var obj = country.toPlainObject()
, id = obj.id;

delete obj.id;
index[id] = obj;

return Object.freeze(obj);
}).then(function (countries) {
log.debug('Loaded ' + countries.length + ' countries');

_countries = countries;

return Geo.Carrier.Descriptor.find().map(function (carrier) {
var obj = carrier.toPlainObject();

if (obj.country) {
obj.country = index[obj.country];
}

return Object.freeze(obj);
}).then(function (carriers) {
log.debug('Loaded ' + carriers.length + ' carriers');

_carriers = carriers;
});
}).finally(function () {
_ready = true;
});
}

reload().catch(function (err) {
log.crit({ message: 'Could not load constants', reason: err });
process.exit(-42);
}).done();

module.exports = {
reload : reload,

ready : function () { return _ready; },

countries : function () { return _countries; },
carriers : function () { return _carriers; }
};


utils.js



var log = sysLog('core', 'utils')
, constants = require('./constants');

module.exports = {
getCountryByISO: function(iso) {
if (!iso) {
return;
}

if ('string' != typeof iso) {
throw new Error('getCountryByISO requires a string');
}

if (!constants.ready()) {
throw new UnavailableError('Try again in a few seconds');
}

switch (iso.length) {
case 2:
return _.findWhere(constants.countries(), { 'iso2' : iso.toUpperCase() });

case 3:
return _.findWhere(constants.countries(), { 'iso3' : iso.toUpperCase() });

default:
throw new Error('getCountryByISO requires a 2 or 3 letter ISO code');
}
},

getCarrierByCode: function(code) {
if (!code) {
return;
}

if ('string' != typeof code) {
throw new Error('getCarrierByCode requires a string');
}

if (!constants.ready()) {
throw new UnavailableError('Try again in a few seconds');
}

return _.findWhere(constants.carriers(), { 'code' : code });
},

getCarrierByHandle: function(handle) {
if (!handle) {
return;
}

if ('string' != typeof handle) {
throw new Error('getCarrierByHandle requires a string');
}

if (!constants.ready()) {
throw new UnavailableError('Try again in a few seconds');
}

return _.findWhere(constants.carriers(), { 'handle' : handle });
}
};


Use case



carrier = utils.getCarrierByHandle(data.handle);

if (_.isEmpty(carrier)) {
throw new InternalError('Unknown carrier', { handle: data.handle });
}


What's going on: All errors are logged; as soon as I see an error (i.e. "Unknown carrier") in the logs, I check the SQL database to see if it should've been recognised. That has always been the case so far, so I check the debug log to see if data was loaded. I always see "Loaded X countries" and "Loaded Y carriers" with correct values and no sign of "Could not load constants" or any other kind of trouble.


This happens around 10% of the time I start the application and the problem persists (i.e. didn't seem to go away after 12+ hours) and seems to occur regardless of input, leading me to think that the data isn't referenced correctly.


Questions:




  1. Is there something wrong in constants.js or am I doing something very obviously wrong? I've tried setting it up for cyclical loading (even though I am not aware of that happening in this case).




  2. Why can't I (sometimes) access my data?




  3. What can I do to figure out what's wrong?




  4. Is there any way I can work around this? Is there anything else I could to achieve the desired behaviour? Hard-coding the data in constants.js is excluded.




Additional information:




  • constants.reload() is never actually called from outside of constants.js.




  • constants.js is required only in utils.js.




  • utils.js is required in app.js (application entry); all files required before it do not require it.




  • SQL access is done through an in-house library built on top of knex.js and bluebird; so far it's been very stable.




Versions:


Node.js v0.10.33


underscore 1.7.0


bluebird 2.3.11


knex 0.6.22





Aucun commentaire:

Enregistrer un commentaire