Some Days I Wish For an Async String.replace
Monday, 19th November 2012, 12:59

The big project I've spent much of my free time on this year, has been moving a mammoth code base from a Windows based ASP JScript to a Linux Node.js one. Part of the reason this has taken so long, is I keep having to stop and do work that pays better in-between.

Another reason is the chance to redesign a lot of how it works under the hood. Like all products, every now and again it really helps to start again from scratch, and just totally redo everything, that way you get to keep the good and dump the used-to-be-good-till-things-outgrew-it.

But possibly the major reason things take so long, is Node.js is asynchronous. It changes so much how things are done, because what was once a single route through your code from start to finish, no longer is. The great thing about doing things async is you don't have to wait for any one part of your app to finish before you get going with the other bits.

The most annoying thing about it, is when the other bits you want to do are reliant on waiting for the earlier bits. Once you get into the mindset, a lot of this is easy enough, but occasionally... just occasionally you hit a snag which makes it all a tiny bit more frustrating.

I hit one of those last week, and it was all to do with text substitution on the forums. This is the second time I came across this issue, the first time I could do a temporary solution that had no effect on the user, this time, not so lucky.

Text Replacing from a Database?

Put simply, the forums on the MyReviewer website use a form of BBCode, the sort of thing I'm sure you've seen in forums where [b] gets changed into a bold tag, that kind of thing.

But there are also some tags which are used to include things like polls into threads. But there is a clear difference between inserting a poll into a thread and a bold or image tag, it's a dynamic structure that requires hitting the database, which means doing an asynchronous call.

Easy enough right? Well not as easy as you'd think at first, because this is how things worked before:

    message = message.replace(/\[poll]([0-9]+)\[\/poll/ig, function($0, $1) {
        return nonasyncPollFunction(parseInt($1, 10));
    });
    
    htmlpage += "<p>" + message + "</p>";

How do we replace a string like that when our pollfunction needs to execute a database query and will return before it's complete? The answer is, we can't. So a few solutions present themselves, one is to create an async version of String.replace which admittedly would be an elegant way of doing it from a design point of view.

But I don't really like that idea because every time I think how to do it, when it comes to global replaces it would require a check to see if the regular expression matched, and then you'd have to run the regex again to do the actual replace, plus it means you can't really do anything with the resulting message string until it's complete.

I Did Say MinnaHTML to the Rescue Didn't I?

And then I realised, I already use the HTML library which has a built in solution for this kind of thing. It has a facility to include child objects which is ideal for this problem, and it's async aware to boot. So now the above becomes:

    var divMessage = new mh.Paragraph(divMessageLine);
    
    message = message.replace(/\[poll]([0-9]+)\[\/poll/ig, function($0, $1) {
        var intPollIndex = parseInt($1, 10);
        
        divPoll = new mh.Div(divMessage, "poll" + intPollIndex, null, true);
        asyncPollFunction(intPollIndex, divPoll);
        
        return "[[child_poll" + intPollIndex + "]]";
    });
    
    divMessage.data.content = message;

And the asyncPollFunction marks the passed divPoll notReady(), then performs the database look-up, makes the poll HTML and then sets divPoll isReady() when it's done. When MinnaHTML processes the document tree to create the final HTML, it automatically replaces the [[child_poll1234]] tags with the poll1234 children objects.

Job done, home for tea and biscuits.