Reid Burke

Reid Burke

  • Stream Concat Anti-Pattern

    NodeConf 2013 was great. You should go. I learned a lot of new stuff. Yesterday, I applied things I learned there for the first time.

    I often write code that looks like something like this:

    var stdout = "",
    child = cp.spawn(process.execPath, args);

child.stdout.setEncoding("utf8");
child.stdout.on("data", function (chunk) {
    stdout += chunk;
});
child.on("close", function () {
    cb(null, stdout);
});

    I want to buffer stdout data that comes from the child process until it exits. Typically, I do this by appending data as it comes in (chunk) to a string variable (stdout).

    This is an anti-pattern. There’s no need for the data I’m accumulating in stdout to be a string. When I convert Buffers into strings with setEncoding("utf8"), and then append them to create an even bigger string, I’m needlessly copying data into V8 when it’s more efficient to keep it outside.

    We want to keep as much data as we can inside of Buffers, which corresponds to memory outside of the V8 heap. Node makes this easy. Since I’m collecting a lot of data here, I could do this better by rewriting that code to the following:

    var stdout = [],
    child = cp.spawn(process.execPath, args);

child.stdout.on("data", function (chunk) {
    stdout.push(chunk);
});
child.on("close", function () {
    if (Buffer.isBuffer(stdout[0])) {
        cb(null, Buffer.concat(stdout));
        return;
     }
     cb(null);
});

    Instead of taking large strings and appending them to enlarge another larger string, I can simply gather all of the Buffer chunks in an array until I’m ready to concat them all into a single large Buffer.

    If I needed to read part of the output, perhaps the last few characters, I can use the Buffer’s slice method to convert a small part of the total results to a string for inspection.

    A smart way to do this easily is to use the concat-stream module, which does this all for you.

    If you liked this, you’ll love going on substack’s stream-adventure. You’re guaranteed to learn something, so give it a try.

  • Do For One

    Sometimes I cruise around on my Tikit. Yesterday I didn’t fare too well.

    San Thomas El Camino Intersection

    While making a left turn, my tiny 16″ wheel sank into the ground at 15 mph. The pothole blended in nicely with years of oil stains from years of cars waiting to turn left. I couldn’t keep the bike under control with one hand signaling left. Next thing I knew, I was on the ground.

    I banged up my knee and had some scrapes. Nothing too bad. But what surprised me was that the driver of the car next to me immediately got out and helped me off the ground. He insisted on giving me a bottled water. He made sure I was okay before taking off. Another woman who saw me limping away stopped her car to give me bandages. I was stoked to see these strangers help me out. But as an introverted guy, inside my car driving by, would I do the same?

    Cars, phones, earbuds, the iPad I’m typing on: All of these things can be world-canceling devices. That isn’t always bad, of course. But it’s easier to miss the opportunity for better use of our time. Jonathan Safran Foer, in his NYT article How Not to Be Alone, shared his experience using a phone to avoid a difficult moment with another person:

    The phone didn’t make me avoid the human connection, but it did make ignoring her easier in that moment, and more likely, by comfortably encouraging me to forget my choice to do so. My daily use of technological communication has been shaping me into someone more likely to forget others. The flow of water carves rock, a little bit at a time. And our personhood is carved, too, by the flow of our habits.

    …everyone is always in need of something that another person can give, be it undivided attention, a kind word or deep empathy. There is no better use of a life than to be attentive to such needs. There are as many ways to do this as there are kinds of loneliness, but all of them require attentiveness, all of them require the hard work of emotional computation and corporeal compassion.

    While we must balance the time we spend on others with time we spend on ourselves, the best parts of my life are those spent helping others. I find myself leveraging modern tools to ignore others, almost by habit, missing the best use of my life.

    The nice strangers who took time out of their day to help me, garnering ire from the traffic behind them, are inspirations to a life-long introvert like me who would struggle to decide to open my car door.

    We’re carved by our habits. Our big decisions are made by all of the little choices we make. Nobody will ever see most of these choices, but that’s okay. Deciding to engage with a messy world in small things will carve out an attitude of compassion that can make a big difference over time.

    You can’t help everyone, but why not do for one what you wish you could do for all?

  • I’m Going to Zimbabwe

    Important update, May 9, 2013: Due to political unrest, this trip will not be happening this year and is tentatively rescheduled for 2014. All funds raised will be put toward a future trip. The original post is below.

    I am going to Zimbabwe for two weeks this July to serve alongside Hands of Hope Africa as they care for orphans and children at risk.

    Zimbabwe has more orphans per capita than any other country in the world. There are 1.4 million kids under the age of 17 who are orphaned.

    I’ll be working at the WestGate Haven Home, which is home to 12 school-aged girls who lost their parents and other close family members to AIDS.

    Girls at WestGate Haven

    I’m going to be with about a dozen other people traveling from San Jose, CA. It’s a huge privilege to be able to go and experience the world in a totally different context.

    After I visited Peru last year, the trip left a lasting impression that made a difference in my life back home. We took deserving kids on a countryside retreat for 3 days—swimming, wheelchair races, horseback riding—which left a lasting impression on the kids and staff too. That makes me excited to serve again.

    Learn more about my trip and why I’m going. Since announcing the trip yesterday on Twitter, I’ve already received a few donations toward the trip’s $4,000 cost, which is fantastic! Thank you.

  • The Real Meaning of Friendship

    Douglas Gresham remembers the friendships of C.S. Lewis in the 1920s:

    Now friendship in those days was a bit different from what it is today; friends did not have to agree on everything and often agreed on practically nothing. They were people with whom you could argue all day and yet never get irritated or angry at all. In today’s world we seem to have lost the real meaning of friendship. If someone disagrees with us, it is fashionable today to dislike them for it. This is silly and robs us of the best kind of friends we could find, for if we are always agreed with, we can never really have a serious conversation; we cannot learn from someone who agrees with what we say.

    Don’t filter bubble your friends.

    Fediverse Reactions

  • Build Modules, Not Examples

    I’ve been thinking out loud this afternoon about how modules make Node.js successful with @ekashida, YUI’s newest team member.

    YUI is a powerful open-source library for developing web apps. Node and YUI both document their APIs very well. Unlike Node, we also have 250 examples to help people get started. Yet lots of people have trouble getting started anyway.

    How do we make that easier?

    We’re not the only web app library with this problem. Ember has received criticism around getting started and quickly responded with a roadmap to improve it. In their response, Tom Dale said:

    Ember promises—and, we think, delivers—tremendous value. But ramping up to that point is not easy, and we received this feedback repeatedly and take it very seriously.

    We get the same feedback for YUI. We also take it seriously and a fellow team member is working on better documentation. Ember’s action items include a Getting Started guide, a short screencast, live examples to run demo code, and other things. These are good and will help.

    But then, I think about Node. Their official newbie documentation begins and ends with Hello World on their homepage. Node’s newbies do not get started with an example. They get started with npm.

    Modules Are The Example

    Node has effectively outsourced their examples to the community using npm.

    You may be thinking: “Reid, modules are not examples. They are tools you use to build an example, or a real app.” Well, you’d be right. Node’s community has built enough modules to help people build apps they care about with a tiny learning curve. Instead of reading about code, these newbies are building something with a tool—the module—that lets them start building what they care about right away.

    Few newcomers to Node.js use require("http") to build their first web app. If the barrier-to-entry was to understand the HTTP module, a lot of people would walk away. They don’t want to learn about that module, they want routing. View rendering. You know, a web app.

    The example is Express. Newbies have a project in mind and it’s probably something that needs a router, view rendering, and middleware. Express delivers that. You don’t need to learn about Node yet, you learn Express.

    Once you’re up and running, you have real code and real problems. Not theoretical problems in a narrative, but problems that are running on your computer. Problems getting in the way of building what you care about. That’s motivation.

    Since you’ve already spent an hour getting the basics setup because the Express API was a lot easier to get started with, you’re now motivated to dig deeper into Node to find the next solution.

    Building A Ramp, Not A Cliff

    Understanding everything Node has to offer isn’t something you learn overnight. But that isn’t necessary. A lot of people have built modules on top of Node’s core that make all kinds of common problems easy. These modules are easy to download and use. Newbies are engaged immediately. Then, they’re are progressively lured into deeper water. The community grows.

    If you need to learn a million things before you can get started, you’ll never start. If that’s your project’s learning curve, you’re presenting newbies with a cliff. Some people will climb it, some will seek climbing experts to get to the top. Some may have the patience to read a book about climbing, or watch a screencast featuring someone else climbing.

    But most people want to get started today, so they’ll leave and find an alternative way to the top that’s easier to climb.

    Node provides many ramps. They aren’t ramps you merely read about: they’re ramps you walk on, build on, get experience with, and get more complicated as you go up. It doesn’t fix every problem and some people will still be confused. But it works pretty well.

    At YUI, I’ll be encouraging our team to focus on building example modules: the ramps to mastery. YUI is already modular, but that isn’t the same. I’m talking about bundling modules in YUI’s core into a fully-baked solution to a common problem. Other people on my team are hard at work on the first product, and now I know why I’m so excited: it’s a ramp.