There should be more people like James Mickens

Warning: in an attempt to blog more in general, and maybe less seriously so, here’s some lighter material than usual. I hope that at some point the usual material will be preceded by a similar disclaimer, announcing some “heavier material than usual”.

I’m a fan of James Mickens. I wish there were more people like James Mickens (well, turns out that there are, kind of, but that’s beside the point). James Mickens is a man who combines genius, benevolence and humor into a grand holistic theme that will take mankind to new heights.

James looks like this:

james

And he writes like this:

I don’t know all of the answers in life, but I do know all of the things which aren’t the answers, and JavaScript falls into the same category as Scientology, homeopathic medicine, and making dogs wear tiny sweaters due to a misplaced belief that this is what dogs would do if they had access to looms and opposable thumbs.

And like this:

Anyways, my point is that browsers are too complex to be trusted. Unfortunately, youth is always wasted on the young, and the current generation of software developers is convinced that browsers need more features, not fewer. So, we are encouraged to celebrate the fact that browsers turn our computers into little Star Wars cantinas where everyone is welcome and you can drink a blue drink if you want to drink a blue drink and if something bad happens, then maybe a Jedi will save you, and if not, HEY IT’S A STAR WARS CANTINA YESSSSS.

And like this:

These are not the best people in the history of people, yet somehow, I am supposed to stitch these clowns into a rich cryptographic tapestry that supports key revocation and verifiable audit trails

That’s all. Go watch his talks and read his papers:

There’s plenty more, of course, and Goog^H^H^H^HDuckduckgo is your friend.

Advertisements

Hoare’s billion dollar idea

Folks, I propose that we forgive Tony Hoare for his billion dollar mistake. Yes, he invented null, and of course all the trouble that got us in is his fault. Fortunately, he also invented CSP, one of the many formalisms for dealing with multiple things happening at the same time. I believe that CSP may end up being his multi-billion dollar idea, more than repaying his null debt to society.

Complicated Scientific Papers

In the time when Hoare invented CSP (1978), every CS professor came up with his own formal language for roughly the same thing. CSP however, has two particularly fantastic features that separate it from the crowd.

First of all, while CSP itself is pretty theoretical and fundamental in nature, most people combine the CSP primitives to roughly the same higher-level building blocks, and these are pretty easy to understand. Secondly, and most importantly, these building blocks are actually in programming languages! Sure, it took us programmers more than 35 years to figure it out (okok except Joe Armstrong, who did it in 8), but still, it’s there!

I’m talking, of course, about channels and processes like they exist in Go, Erlang, Clojure (core.async) and a bunch of other languages. Nitpickers will immediately complain that Erlang doesn’t really have channels because it follows the Actor model instead, but you can simulate channels with actors and you can simulate actors with channels, so I’m going to completely gloss over that difference from here onwards. The most important part are processes and we have those in all these languages.

Processes are awesome!

The idea of a process is a piece of code that actively does something and occasionally tells others about it. It looks and feels a bit like a thread, except that you don’t have to think about how much a resource hog threads can be. Also, because you’re not supposed to use shared data, you don’t have to think about mutexes and all that other stuff either. You use messages for sharing data.

For example, in a UI application, you could have a process that polls for mouse click events. “Wait”, I hear you say, “don’t mouse clicks usually come from some sort of event loop? Why poll?”. Indeed they do, but the fun about processes is that you don’t really care about that. Even if in some weird UI system, you’d need to actively poll for mouse clicks, you can make the process work exactly the same from the outside.

This mouse-click-poller process could report clicks, with some additional event information, on a channel, and any other process that is interested in mouse clicks and receive messages on that channel. This feels a bit like normal event listening, but the fun is that the mouse-click-poller can also do some other processing on the event. Maybe make the event object easier to consume for listeners, do some post-processing or cleanup work, and the listeners don’t need to change how they work at all.

This is fantastic separation of concerns, and allows for excellent composability. You could have a process that listens on channels of mouseup, mousedown and mousemove events and produces drag events on another channel, for instance.

Cute little robots with opinions

Many other programming languages allow you to do similar things. For example, C# gave birth to the excellent Rx library and has fancy language-level async/await support which allow you do so a similar thing. In fact, composing multiple lower level mouse events into drag events is sort of the “Hello World” of Rx. As another example, you can’t call yourself a competent JavaScript developer these days without embracing promises or streams something comparable. These are all great tools for dealing with asynchronicity and concurrency, but I feel like they all, at least in some ways, make a program more complex to understand. I want my program to be simpler.

Being able to treat everything that has some sort of active behavior to it as a separate process is a truly liberating experience. I first had it when learning Elixir, a language based on Erlang, in which such processes are a native feature. Many programmers had it when learning Go and writing their first goroutine. I find that “behavior”, as vague a concept as it is, is a much more important thing than “asynchronicity”. Therefore, I believe that is not the right solution to add language level support for things like “async methods” (C#) or “async return values” (JavaScript promises, Scala futures, etc). As a caller of functions, I shouldn’t really care whether something is async or not, I should care whether I’m interfacing with this independent, active thing over there (like a cute little robot with an opinion) or a simple function that turns some input into some output (like the calculator on my desk) (okok the calc.exe in my Run dialog). Cute little robot? Process. Calculator? Function.

The world’s most popular programming language

You can imagine my happy joy when I finally got around to reading James Long’s excellent blog post on putting CSP processes in JavaScript. If you do any frontend coding, do yourself the favor of at least skimming through his post. To me, this style of programming feels so much better and cleaner than promises or callbacks or all that mess, that I can hardly fathom it. Fair enough, it requires ES6 generators, which in turn require either hyper-modern browsers, or a Node.js fork built by irresponsible hippies, or compilers like Regenerator which produce horrible JavaScript. But hey, we have source maps anyway, so who cares about what the browser sees?

I truly believe that the future of anything that has asynchronous or concurrent behavior, which is nearly any piece of performant software really, lies in the concept of CSP-like processes. Processes are to futures, tasks, async/await, and promises what garbage collection is to malloc() and free(). The latter burden the programmer with, in my eyes, unnecessary implementation details in the vast majority of cases. I want to organize my program into a number of cute little robots, not into a big bag of functions that may or may not have asynchronous behavior and oops you forgot to catch() that promise and what the hell just happened in which order again?

Processes are the future, futures are not, and it is now.

I am going to use JS-CSP in all my future frontend projects. I’m going to avoid languages on the backend that don’t either natively have processes or have a great library that supports it. Dear C#, I truly love you, but you’re out the door for me until you get your concurrent act together. Processes are great, all the other stuff is not, and I have CSP to thank for it.

Is CSP going to be Tony Hoare’s billion dollar idea? I think that maybe it already is.

Named parameters for humanity!

I had a little insight today.

Many modern languages, including my current favourite toy C#, support passing parameters by name. For example, consider a method like this:

public State MoveRobotArm(
    double position, 
    bool throwOnErrorState = true, 
    bool simulated = false)
{
    ...
}

Notice how its last two parameters have default values, which means we can omit them if we want. We can now call this method like this:

    var state = MoveRobotArm(50.0, simulated: false);

We’re skipping the second parameter and assigning a value to the third. This is what named parameters were intended for. However, it’s also remarkably readable! Now, if you came across code like this:

    MoveRobotArm(50.0, true, true);
    var state = MoveRobotArm(0.0, false, true);
    if(state != State.OK) ...  // handle errors

Would you be able to guess what the MoveRobotArm calls do? There’s some things to deduct. The number is probably a position, and given that we throw away the return value of the second MoveRobotArm call, we’re either crappy coders or one of the booleans help manage unexpected situations.

Recently, I started writing such code like this instead:

    MoveRobotArm(50.0, throwOnError: true, simulated: true);
    var state = MoveRobotArm(0.0, throwOnError: false, simulated: true);
    if(state != State.OK) ...  // handle errors

It’s a minor difference, but it makes the code much more readable. You don’t need to look up the signature of MoveRobotArm to figure out what’s going on.

The self-documenting power of named parameters works especially well when booleans are used as flags. You could also write an Enum for each option and have it be equally self-documenting, but that feels like a lot of boilerplate. This is the same, but with comparably little boilerplate. Similarly, when methods take many parameters of the same type, it is very helpful to the reader if the parameters are named on invocation.

Down the rabbit hole

Note from the editor: That was the end of the blog post. The rest is just mindless rambling.

I’m starting to think that it would be a nice idea for some new programming language to force callers to name arguments be default. I think it’d make 80% of code more readable (and hardly more cumbersome to write, assuming you have a decent IDE). But not always: you wouldn’t want to have to write Math.sin(value: 5.0) for instance, it’s ridiculous.

So you need an escape for allowing unnamed parameters, but we can leave it up to the author of the function to decide whether to allow that or not, using some extra syntax.

Essentially, for most current programming languages, function parameters are an ordered list of mixed types. I’m proposing that it’s, by default, an unordered collection of name-value pairs (i.e. a struct, object, record, whatever). You can then have syntax to allow one or more unnamed, ordered, parameters as well, if you really want to. You’d get maybe something like:

    void CallMyMother(int phoneNumber, string message) ...
    // OK:    CallMyMother(phoneNumber: 123456, message: "Hi Mom!");
    // OK:    CallMyMother(message: "Hi Mom!", phoneNumber: 123456);
    // ERROR: CallMyMother(123456, "Hi Mom!");

    string Diff([string a, string b], bool caseSensitive = false) ...
    // OK:    var diff = Diff("aaabb", "aaacc");
    // OK:    var diff = Diff("aaabb", "aaacc", caseSensitive: true);
    // ERROR: var diff = Diff("aaabb", "aaacc", true);

    double Sin([double value]) ...
    // OK:    var slope = Sin(2.0);
    // ERROR: var slope = Sin(value: 2.0);

Something like that. I’d like that.

IoC containers: prozac for software architectures

I think Inversion of Control (and Dependency Injection, basically the only decent way to do IoC in OO languages) is a great idea. We do it all over our Izooble codebase, for example: it enables testability and encourages us to keep independent things separated. I believe that Dependency Injection is one of the few good design patterns that still make sense in modern OO languages. I suspect that it, too, will at some point be superseded by a programming language designer’s great idea, but right now I can’t imagine what that would be.

What I do think is wildly overrated, is IoC containers. I don’t see their added value over explicitly injecting a dependency into a class’s constructor. It’s the singleton antipattern in disguise, and with an IoC container it’s very difficult to see what is going on, because, magic. It’s even worse when the IoC container is configurable with something other than code (e.g. Spring.NET’s horrible, horrible, horrible XML abomination), because that actively encourages messing around with live servers* instead of fixing your damn deployment pipeline.

If you avoid IoC containers, you’ll typically end up with a Main method that instantiates a whole bunch of objects and passes them to other objects again. That Main method is an excellent summary for your entire application’s architecture. You can see how all the important components are tied together at a single glance.

Now, when your application grows, when you do this, two things tend to happen:

  1. Some class constructors, those high in the composition hierarchy, may need a very large amount of injected constructor parameters, just to pass them through to a class way down the tree. Well that’s cumbersome.
  2. Your Main method becomes a ginormous 100-line list of new operators.

Somehow, people take this as a sign that they need an IoC container, so they can hide the monolithic, highly coupled contraption they inevitably grew their application into.

The real solution, of course, is to fix the architecture. Split the application into multiple independent components that only have few dependencies on each other. In fact, you can use a nearby whiteboard to trivially turn your Main method into a tree diagram so you can see how to best cut your application into pieces. Maybe you’ll need to reverse some dependency arrows here and there (events to the rescue – or Observer if it’s 2002 and all you have is Java). You’ll end up with a bunch of independent parts, with the old content of Main spread across constructors of Facade-style entry point classes in these parts. Congratulations, you just forced yourself to fix your architecture by stubbornly avoiding IoC containers.

I really don’t understand why some programmers insist on hiding their bad designs from their own view. It’s like they don’t want to know that things are becoming a mess. IoC containers are to architecture what .NET regions are to classes.

* Or robots, or machines, or whatever your software runs on.

Leaving .NET is a clueless thing to do

Every so often, someone writes an extensive blog post about why they left the .NET framework for greener pastures.

Invariably, the blog post contains complaints about other .NET devs’ inability to see beyond Microsoft Wonderland.

I agree with that complaint. Indeed, many people who code C# are wildly uncomfortable using open source projects, text editors, command lines, and so on. This is a problem and it’s bad for the .NET ecosystem.

What people should be doing, instead, is use the Microsoft goodies when it makes sense and something else when it doesn’t. For example, instead of using ASP.NET MVC, it’s perfectly possible to make a web app with a C# backend running on Mono, talking to a Postgres server and a custom Javascript frontend. Nobody says that if you code on .NET, you have to do it exactly like Scott Hanselman does it.

Similarly, when C# or .NET are simply not the best tools for a job, don’t use them. Quick prototypes may be much easier to churn out in Ruby. Self-hacked TCP protocols might be much easier to serve with Node.JS.

What itches me is that all these people who complain about this in their .NET-post mortems are doing exactly the same. What in the world does it mean to “leave” .NET? I didn’t code Python in the last 5 months. Does it mean I left Python? Should I now write an annoyed blog post about all the things that are wrong with Python and its community?

Of course not – despite its shortcomings, a lot of stuff is good about Python, and exactly the same holds for .NET. In fact, as Microsoft itself is opening, the navel-gazing parts of the .NET ecosystem may very well follow, at some point.

C# is great for many purposes. For example, with C# 5’s async/await syntax, it’s one of the best options out there for high-performance asynchronous code. Just like Go has become a go-to language for high concurrency, C# may very well become a go-to language for high asynchronicity. Don’t ditch it just because an earlier career mistake means you never want to write WebForms again.

Cool chemistry projects!

GitHub launched a new thing called GitHub Education. I bet it’s really cool, but that’s not what I want to talk about today. The front page looks like this:

GitHub Education blackboard with cool chemistry set

See that chemistry set there on the left? It looks so awesome! With tubes and big spherical bowls and steam and fire and bubbles, wow! Can I send my kids to a school that has that?

School projects in media are often drawn like this. Sherlock Holmes has a similar setup in the middle of his kitchen. But has anyone ever done anything that remotely resembles that in school? I know for sure I haven’t. Our chemistry projects looked like this:

Boring bunsen burner with test tube

Indeed, a test tube. Ok, and some heat, so at least we had danger. But still, boooring.

And, that makes me think, why not? Wouldn’t chemistry class be a lot more fun if we could make it look cool? Even if the experiment doesn’t need for some fluid to flow from bowl A to glass B, can’t we still make it? Maybe this is just the LEGO fanboy in me talking, but I feel like there’s an opportunity here!

String types are fine. How about your code?

Someone on the internet says that strings are broken, and some more people on the internet disagree.

The problem is that all kinds of common operations on strings, such as counting the amount of characters in a string, or converting a string to upper case, become a horrible mess when you want to support all the characters in the world (and then some). People discuss how well their favourite programming language solves this problem for them. The conclusion seems to be that no programming language does it perfectly, because that’s impossible without additional information (such as which language the string is in, and who it is for).

Text is for humans

I daresay the entire discussion is misguided, because strings can be used for different purposes, and you shouldn’t mix those purposes. In code I’ve seen, there’s three categories of string use:

  1. Strings intended primarily for machine consumption (JSON keys, enum values being sent over the line, dynamic method calls, URLs, etc)
  2. Strings intended primarily for human consumption (UI texts, user input such as comments on a blog, names, and so on)
  3. Strings used for both at the same time

Category 3 is the problem. That’s where this entire discussion comes from. A programmer got names of places or people, titles of texts, phone book entries, and turned them into identifiers. He wants to ensure that if another user types the same word again, the same thing is found, so he thinks, “I know! I’ll lowercase everything and replace non-alphanumeric characters by ‘_'”. This goes fine until someone enters “ანბანი”.

Hashing prose

What’s really going on here is that the programmer got text from category 2, and wanted to transform it into something from category 1. This is perfectly OK, as long as you stick to two rules:

  • You can only go from human-only to machine-only, not back. Essentially, you’re writing a hash function.
  • Like with any hash function, you need to think about its uniqueness properties. If you need uniqueness but cannot reasonably it (which happens very quickly once you start converting characters to ‘_’), you need an additional unique identifier. This may have security implications, too.

Any code that does not follow these two rules automatically ends up in category 3, which is a code smell.

Transforming text from category 2 to category 1 is a rather common operation. For example, maybe you want to derive a pretty blog post URL from its title. If a user might write about C one day, and about C++ the other day, you either need to keep the pluses in the URL and end up with “C%2B%2B_rocks“, or you need additional information. This is why most blog and newspaper URLs contain text and an identifier.

Google does something similar when you search for misspelled words. “Łódź” doesn’t sound like “Lodz” at all, but Google doesn’t care, to great joy of the Łódź tourist board and all Poles who find themselves behind a non-Polish keyboard. Google needed to support a near-perfect conversion from category 2 (user input) to category 1 (indexed keywords). Because this is impossible, Google accepts that sometimes you get results that you don’t want.

It’s a one-way street

Any attempt to predictably go from machine-only text to human-only text is futile. Once a string has turned into an identifier, don’t try to get the original back. It’s a hash, you lose data. You may be able to find multiple human-readable texts that match a single identifier (such as Łódź and Lodz), and this might be useful if you’re building a search engine. In many cases, just don’t try.

The fun thing about this one-way street is that it’s a one-way street that you control. Whatever the user enters, you can set the assumptions for any strings that fall in category 1. You can ensure that machine-only strings contain only alphanumeric characters, or only ASCII characters, or only valid identifiers for your favourite programming language. You can clearly set these assumptions and then work with them. Once you work with a limited string alphabet, you can go wild on substrings and lowercasing and comparisons and all that, without much going wrong.

There is no requirement that JSON keys contain only [a-zA-Z_-]. Yet, nearly everybody appears to stick to this convention. Why? It’s machine-only data, so no need to make things complicated. String types in nearly every language, even ones with horrible Unicode support, are fine for use in category 1. Go wild! Strings are fine!

Human-readable text should not be touched.

In an odd kind of duality, there is often little need to change or analyse strings that fall in category 2. If you have a user interface in many languages, don’t “intelligently” uppercase words. It’ll go wrong. Have translators produce a string for both “Ok” and “ok”, if you need both. Trust people, not brittle string classes. Similarly, don’t be smart about transforming words, names and sentences that users input. Unless of course this is a core aspect of your product, like when you’re coding Google Maps and you want users to be able to search for both Tokyo and 東京.

For human-only text, you don’t want to do manipulations or analysis. Need a string length for correctly rendering a UI? You can use String.Length, but be aware that it can produce inaccurate results. If you need to be sure, you need to use the underlying rendering library and measure pixels or centimeters and not characters. Similarly, why would you ever need to take a substring of someone’s name, or a poem, or the Russian word for “Banana”?

The moment you feel like you need to perform these kinds of operations on strings for humans, there might be something the matter. Probably, you need to hash the string first. Go to machine-only strings.

If human-only strings, however, are only read and then displayed, then any sufficiently modern Unicode-supporting string class suffices, again. Indeed, yet again, strings are fine. Go wild!

Built-in datatypes

If there is any takeaway from this entire discussion, it may be that there is a need for multiple string types in strongly-typed languages: one for machine-only text, and at least one for human-only text. Such a human-only string could contain no common string operations at all, except for converting from and to byte streams in various encodings. Similarly, UI frameworks and template engines could make it difficult to display machine-only text, just like how modern HTML template engines help avoid XSS attacks.

Note: I read on Hacker News that Ruby actually does something like this: it has one class per encoding. Declare a law in your Ruby shop that ASCII strings (plain old Ruby strings) are to be treated as machine-only, and you’re pretty far.

Epilogue: There’s no free lunch

Unfortunately, all of the above holds until you want to print a phone book for the entire world. Does Орёл sort before or after Oryel? They’re the same place name, just written differently. Any sort of human-understandable sorted list of things written in multiple languages gets really messy real fast. Fortunately, phone books have been largely replaced by search, and if you accept some false positives, search works better anyway. And always go to category 1 when searching.

Of course, if you only need ordering for some internal algorithm you have, you can probably afford to go to category 1 first. If not, maybe the actual ordering does not matter, as long as it is consistent.

If you got this far, you’ll probably want to hire me as a consultant.

Note to self: Always start with a contract

I started a one-man consultancy in September, and this week I ran into my first “situation”. I had already started working for a customer while we were slowly working out wording of a little 2-page contract. We had verbally agreed on a rate and confirmed this per email. Indeed, verbally. I guess that you can imagine what’s coming next.

So, woops. This week I learned the hard way that as an independent, I should not do significant amounts of work for a customer before the main points of an agreement have been put on paper and signed by both parties. Things have been resolved since, to the satisfaction of both parties, so I will spare you the details. Nevertheless, I did have quite a scare.

Trust

In practice, if an independent does contracting for a larger company, there is little real legal benefit to having the contract worked out quickly. After all, if things would really come to a legal conflict, I am an inexperienced single entrepreneur with a relatively urgent need to feed a family, while my customer typically has more legal experience, more business experience, and more patience. The best possible outcome of a court case will be that both parties lose, instead of just me.

In such an asymmetric relationship, I need trust, not paperwork. Because of this, a contract does not actually provide much real legal support. Instead, it is there to help ensure that all people involved at my customer’s organization agree to the terms. It increases the chance that whoever will pay my first invoice is also the person who signed the contract, and that all bureaucratic and company-political hurdles have been overcome. There’s no certainty until that first invoice has been paid, but I can’t usually demand payment on day 1. I can, however, demand a signed contract.

Using time? Inject a clock.

Your favourite language’s way to get the current time (e.g. new Date() in JS or DateTimeOffset.Now in C#) is lovely. It’s also a nasty global singleton, in that you can’t mock or stub it from tests.

The problem with directly referring to “the current time” from application code is that you often end up doing Thread.Sleep(...)s all over your automated test code. This makes tests slow and brittle. Now, an automated test with some busy waiting an ugly thread magic is better than no test at all, but if you have the chance, better avoid it.

I found that, in nearly all cases, it helps to inject a clock:

public interface IClock
{
    DateTimeOffset Now { get; }
}

This interface will allow you to very easily simulate time-derived behaviour. I usually use a RealClock that the application uses, and a FakeClock used by tests. They’re ridiculously simple too:

public class RealClock : IClock
{
    public DateTimeOffset Now { get { return DateTimeOffset.Now; } }
}

public class FakeClock
{
    private DateTimeOffset now;

<pre><code>public FakeClock(DateTimeOffset startTime)
{
    now = startTime;
}

public FakeClock()
{
    now = DateTimeOffset.Now;
}

/// &amp;lt;summary&amp;gt;
/// Gets or updates the clock's current time.
/// &amp;lt;/summary&amp;gt;
public DateTimeOffset Now
{
    get { return now; }
    set
    {
        if (value &amp;lt; now)
        {
            throw new InvalidOperationException(&amp;quot;Can&#039;t decrease time.&amp;quot;);
        }

        now = value;
    }
}
</code></pre>

}

The FakeClock could have been a lot simpler, but I choose to enforce that time never rewinds. Code assumes this more often than you'd think, and there's nothing wrong with that.

Using an injected clock, you get real nice test code, for anything that depends on time changes, such as UI animations or hardware simulators.

Imagine a screen that has an alarm trigger, which when activate is on for exactly 5 minutes:

[Test]
public void AlarmShouldBeVisibleFrom5Minutes()
{
    var clock = new FakeClock();
    var screen = new SomethingScreen();

<pre><code>screen.ActivateAlarm();
screen.Alarm.Enabled.ShouldBe(true);

// verify that the alarm is still going 1 second before
// the 5 minutes have passed.
clock.Now += TimeSpan.FromSeconds(299);
screen.Alarm.Enabled.ShouldBe(true);

clock.Now += TimeSpan.FromSeconds(1);
screen.Alarm.Enabled.ShouldBe(false);
</code></pre>

}

The key point is, of course, in code like clock.Now += [something];. Even if you hate mutable state, you have to accept that time is inherently mutable. Simply updating the time from automated tests allows for a pretty simple way to deal with that.

First!

Well, it’s almost 2014, I heard that blogging is the hip new thing young people do on the internets, so here goes! The idea of this blog is to not hold back to much for writing things. Anything that doesn’t fit in a tweet, goes here.

I chose WordPress because they support pasting code, see:

parse read http://www.rebol.com [
  any [
    thru "A HREF=" copy link to ">" (print link)
  ] to end
]

Code. Lovely.

They do markdown too, actually. And pretty decently, including all the Github Flavored Markdown goodies n’all. nice!