NodeJS 0.8.x May Leak Your Environment Variables

NodeJS up to 0.8.x will, under some circumstances dump your environment variables into a file which you may inadvertently check into source control. If you have sensitive information stored in your environment, it will be exposed when you publish the repository.

If you've used NodeJS to build any of your projects, particularly if they're open-source, search for a file called .lock-wscript under your node-modules directory. If it exists it will probably contain a dump of your environment variables.

This happened to me and I only found out about it when Amazon notified my client that her AWS credentials had been compromised and gave her a link to a file under my Github account. Not a great look. Fortunately, that gave me a chance to invalidate the credentials before any harm was done.

Since then, I've taken the following steps:

• 23 Apr 2014 - Reported the issue to cert.org (they declined to publish it)
• 24 Apr 2014 - Emailed security@github.org (received no response Github will add .lock-wscript to their standard .git-ignore)
• 24 Apr 2014 - Emailed security@nodejs.org (who have responded with a security patch)
• 6 May 2014 - Published this blog entry and notified the NodeJS mailing list

Fedor from the NodeJS team responded with this patch. I haven't tested it because I'm not actually using 0.8.x any more and, to me, the real issue is that there are existing published repositories with people's environment variables dumped in them.

Should it matter?

The existence of this file isn't news. Presumably a lot of people knew that it contained environment variables and didn't think it was a problem. Others, including myself, were under the mistaken impression that environment variables are private and are a reasonable place to store sensitive information such as security keys.

The combination of these two assumptions (by separate parties) can, and in my case did, lead to sensitive data being exposed.

For my part, if I do use the environment to ship sensitive information around in future, I won't be exporting it from my shell.

Recommended Actions

My personal recommendations for what you should do:

 - Check your own repositories for .lock-wscript files and invalidate any keys contained within.

 - Add .lock-wscript to your global .git-ignore

 - Review the environment that your shell is passing to every program you run and move anything sensitive into a more local scope.

 - Tell your friends about this if they might be affected.

Regret

I was listening to this this podcast last night about the psychological effects the Milgram experiments had had on the participants. I have to admit that it had never occurred to me how those people must have felt being equated with Nazi concentration camp guards.

The subjects who had been tricked as part of the experiment into participating in (mock) torture seem to have been dismissed as bad people and sent off to think about it for the rest of their lives. Thinking about it a little further leads me to conclude that the experiments were not only cruel, they were basically pointless because they'd already been done.

Unless we were to resort to the idea that Germany had been overtaken by an unexplained wave of psychopathy during WWII then we'd have to conclude that ordinary people can be made to do terrible things under certain circumstances. Unfortunately, Nazi Germany is far from the only example of this.

I've never been subjected to that kind of experiment but I have been plagued with regret. I have on many occasions in the past caused pain for others or myself through stupidity or fear and part of my mind won't stop reminding me of that. I suspect that regret is a bigger problem than commonly acknowledged. It is the worst part of depression for me because past actions can't be changed.

The first thing to notice about regret though (particularly for acting badly toward others) is that it's a sure sign that you're not a sociopath. Your conscience is working just fine, in fact maybe a little too well. Relief at this thought seems paradoxical though. If you feel bad, you shouldn't but if you don't then you should.

But regret, beyond what is useful for us to learn from and regulate our behaviour, that debilitating regret, is also based on a false premise. That premise is free will. Only if you could have done otherwise but chose to do the wrong thing do you deserve to suffer. Surely if you had no choice, you are blameless. You may need to be corrected so that you do better next time but that's it. There's no need to go on punishing yourself.

Fortunately, with a little thought, you can demonstrate to yourself that this is in fact the case. You had absolutely no choice but to do what you did. Don't believe me? Lets do a little time travel.

Lets go back in time to when you made the biggest mistake of your life (so far :P ). We'll put nature into reverse, return every atom in the universe to the point that it was at, at that moment. Of course, every atom in the universe includes the ones that make up your brain. So there you are with exactly the same set of inputs, the same state of mind. What will you do? You'll do exactly as you did before. By what mechanism could you do otherwise?

If I haven't convinced you, I'd recommend Sam Harris's excellent book "Free Will". In it, he gives free will the sound thrashing it deserves and also explains why even the idea of a soul (if you believe in that sort of thing) can't rescue the concept of free will.

So where does that leave regret? I'd define regret as anger at one's self. And what are you angry for? For doing what you absolutely had to do?

This isn't a license for bad behaviour. Of course we have to constrain our actions. But regret, as with anger toward others, is only justifiable to the extent that it improves us.

So the next time, you're chewing yourself out over past actions, consider whether that's really necessary or if you've already learned. Over time, maybe you'll find that the regret dissipates and you're able to focus on your present actions.

Activerecord Boolean False Gotcha

Unless you've got a good reason not to, when you declare a boolean field in Rails/ActiveRecord, always set give it a default and disallow null. Eg:

 t.boolean "troublemaker", :default=> false, :null => false

The reason for this is that when you want to search on it later to see if it's false, you can simply say:

Student.where( :troublemaker => false)

The above statement won't find students where the :troublemaker is null so it's a good thing you didn't allow that to happen.

If you already have boolean fields in your DB that may have null values which you were regarding as false, it's easily fixed. Just run a migration like:


class DefaultTroublemakerToFalseForStudents < ActiveRecord::Migration

  def self.up
    change_column_default :students, :troublemaker, false
    Student.where(:troublemaker => nil).each {|s| s.completed = false; s.save!}
  end

  def self.down
    change_column_default :students, :troublemaker, nil
  end

end

Well, there's 10 hours I'll never get back

Sunday night, I'd decided to switch back to using Textmate. I'd given Vim a pretty good go. I'd installed a bunch of plugins and even gone as far as writing a bash script to load them when I went to a new machine.
But now I felt like I was spending more time thinking about Vim than I was thinking about the software I was writing. Basically, it just felt like too much mental effort.

Monday morning, I saw a reference to something called Janus. Not sure why I followed it but I'm glad I did. Carlhuda and Matsumoto have done a fantastic job configuring MacVim. They've turned it into the editor that I wanted all along.

They accurately describe Janus as a distribution. They didn't write Vim itself or MacVim or the plethora of plugins that are doing the heavy-lifting. They just bundled it together and made it just so. And they did a much better job than I'd managed to.

So now I don't have to forego the multi-windowed,  ubiquitous, open-source goodness that is Vim to get a beautiful full screen intuitive Rails development environment.

The only additions that I'd like to make (at this stage) are:

• DBExt.vim: This is for performing raw SQL queries (remember those) in the editor and integrates very nicely with Rails.vim. Rails.vim provides a RDBExt command which takes as its only argument the name of an environment in the current Rails project. It creates a configuration such that any DBExt command will execute against the database for the nominated environment.
• Some better integration with continuous testing frameworks like autotest. I'd like to jump to the last error detected with a key combination. This may already be possible. I just don't know how.

I also change some preferences and set up some key mappings in my .vimrc.local below:

This allows me to jump between buffers without having to save edits first (which I find annoying). It also maps some keys to perform some common rake tasks. For example, \sm (assuming \ is your leader character) will run the model specs.

If you're not familiar with it already, gf jumps to the filename currently under the cursor. Rails.vim extends this to be more Rails aware. Here we're mapping gf! to force the creation of the file if it doesn't already exist. Note, if you're too slow with the ! then vim will assume you mean gf and then fail to create the file. I kind of like that element of realtime in my text editor.

Janus has convinced me to stick with Vim, at least for now. It's also taught me to look harder for existing solutions before building my own.

Samsung: Korean for Fail

I've just looked up "Samsung" on Wikipedia. They confirmed that Samsung is in fact a company. I checked because I'd started to believe that it was a long running practical joke.

I've just given up attempting to upgrade my Samsung Galaxy S to Android 2.2 (Froyo). I was hoping that Froyo would resolve some of the deficiencies of the SGS such as:

• Periodic jam ups including a few when I was trying to answer a phone call.
• Failure to automatically switch between 3G and Wifi (the work-around is to switch it to airplane mode and back or if that fails, turn the phone off and on again, what fun).
• Popping up an alert (day or night) to tell you that it's finished charging. Please wake me at 3 in the morning to tell me you've finished charging, I'd hate to sleep through that event.
• Requiring debug mode to connect to a computer running any OS other than Windows.

But of course if you run OS X or Linux you're out of luck anyway because it only provides support software for Windows software for essential operations like upgrading the damned thing. Not that it matters because it has such a shonky support software that it's hit and miss whether it will even recognise that a phone is connected at all. I've just given up trying to get it to work for now having read that many users have bricked their phones just attempting to do the manufacturer sanctioned upgrade.

I read reviews before I bought this thing. They raved about it's screen. Yes, the screen is nice. They pointed out how fast the processor was and I got the one with the built-in 16GB SD card. Having read the reviews, my biggest concern was that it didn't have a flash for the camera.

I know better now. It's the little things that make this such a pain to use. Things that must not occur to a reviewer in the short period of time he or she is using it.

Before I had this I had an iPhone 3G. I liked it. The browser was a bit slow compared to modern phones, it didn't have a compass so it wasn't great for navigation but it worked. There was a load of good quality software. I mean sure not everything in the Apple App Store is a gem but have you seen the Android Market? I'll take the curated experience thanks.

When I went to Android because I thought I'd be keeping all of what I had with the Apple and gaining better Google interoperability. It just isn't so. Samsung has made a powerful case for Steve Job's conjecture that you can't guarantee a quality user experience unless you provide the full hardware and software stack.

Is this completely one sided? No. Android's killer feature (compared to the iDevices) is the "back" button. I think that it's more than having a physical button, it's how the OS works but having a consistent way to go back to the previous app is a big thing and I miss it when I borrow my wife's iPad.

The freedom to choose install alternate input methods is also good (but not essential). Apple seem to resist that level of customization. Their "you don't need it" policy at work. Maybe if I'd bought an HTC Desire I'd be feeling better about the whole Android thing but Samsung have left me more than willing to give up some freedom just to get something that works.

What I find amusing is that Samsung sent a free Galaxy S to a few prominent twitterers who complained about the reception problems with the iPhone 4. I can only imagine that it was to make them appreciate how lucky they really were.

Coding Karma's Gonna Get You

If you're writing a server based web application (in any language) and you're not following MVC principles, then please stop and go Google "Model View Controller".

I've recently inherited a site with pretty much no documentation and a mess of legacy PHP code with some Java and Perl thrown in. Am I having a good time? No, not really. What would have made it better?

Well, it's tempting to say documentation. If well written, that may help. An overview from a business perspective would have been really welcome. But even if I'd had that, it would help only a little to work out how the system itself works.

Every time I try to work out what's going on, I trawl through pages of procedural PHP interspersed with SQL queries to a non-normalised database. It's confusing and exhausting. Please don't inflict this on your successor.

Am I just whinging? Well...yes. But with a purpose. There is a certain amount of inherent complexity in any system but there's no need to complicate it further by not employing tried and tested techniques and mechanisms available to you. If you do, you're doing yourself, your employer and your successor a disservice.

Use frameworks people. I'd recommend Ruby on Rails. But if that's not for you, pick one that is. If your application displays a web page, takes some user input, accesses a database and displays another web page then please use an MVC framework.

Here's what a decent framework brings to the table:

 - DRYness. Don't Repeat Yourself. Copy and pasted code is annoying for the person who has to read it. Copy and pasted code is annoying for the person who has to read it.

For every request that comes in, your application will have to do certain things. Every time it accesses the database, there is a certain amount of common code that gets run. A framework's job is to implement this boiler-plate code and hide it so that your program consists of the things that differentiate it from every other web application.

 - Separation of concerns; If you're doing SQL queries in the same that generates HTML then (seriously) please stop coding now and do some reading. It's quite possible, and highly desirable to separate your CSS, HTML, Javascript, View, Controller and Domain logic into separate files.

 - Testing; Manually testing applications is expensive in terms of time. You really need a batch of tests that run every time you make a change to ensure that no existing functionality has been lost. The best time to write these tests is before you right the code. That's called test driven development and (done right) will save you and your users much time and frustration.

The fewer dependencies it has, the easier code is to test. If you need to setup a web server, database server, and an ActiveDirectory domain to test that your code works you're very likely not to do it at all. This was one of the reasons why separation of concerns (above) was desirable. The separate units can be tested individually without the overhead of bringing up a complete system. Your framework should provide tools and instruction in how to do this.

 - Convention; How much does it matter to you what you call the folder containing your controllers (that's the C in MVC)? Surely very little so long as:

 a. There is one and it only contains controllers.

 b. Its called the same thing and in the same place for every project.

 c. The next person to work on the project can reasonably be expected to identify it.

 - Business Continuity; Not only is the interchangeability of developers essential for the business, it's probably in your interest too. Do you really want to stay in this job forever? Do you really want to the be one person who can debug it? Your system will tie you down to it until someone eventually replaces it and you. A framework acts as a label in a job ad. Yes, I am a SpringMVC developer. Yes, I'm a Django developer. I understand the concepts and conventions employed by this framework. I know where to get help if I need it. I know that I can walk into that job and quickly understand your system.

 - Experience; If your framework is a good one, preferably developed over a period of time with many people in a meritocratic environment then the result will be far more refined than anything you could hope to achieve on your own. By all means, learn from first principles. It's good to understand the lifecycle of an HTTP request by writing a CGI script or even a PHP page. And when you're done, save it in a folder called "Learning" and start using a framework.

Missing images with JQTouch on Rails Production

It's generally a good idea to use the ":cache=>true" argument to stylesheet_link_tag to ensure that all your stylesheets are loaded in a single request in production. Eg:

= stylesheet_link_tag '/jqtouch/jqtouch.min.css', '/themes/apple/theme.min.css', 'mobile', :cache=>true

This can cause problems though if the stylesheet:

1. References resources (such as background images) via relative URLs
   
2. Lives somewhere other than public/stylesheets
   

The reason it's a problem is that the :cache=>true option causes stylesheet_link_tag to bundle all listed stylesheets into /public/stylesheets/all.css. Suddenly, every URL is relative to public/stylesheets.

I encountered this problem when using JQTouch in a Rails app.

The theme stylesheet for jqtouch was located in /public/themes/apple/ and it expected to find all images in an img sub-folder. This worked fine in development but in production, suddenly no images appeared.

My fix for this was simply to create a symbolic link in the stylesheets folder thus:

cd stylesheets
ln -s ../themes/apple/img img

This seems to do the trick, even on Heroku.