Nathan Wailes - Blog - GitHub - LinkedIn - Patreon - Reddit - Stack Overflow - Twitter - YouTube
Erlang / Elixir
Reviews of the language
What I think
- I want to have my thoughts on the language written down for future reference in case I feel a need to decide whether to use it over Python.
- According to Adam D'Angelo in ~2014, "Erlang is good at holding a ton of open connections, which is the hardest part of a chat service. (...) I think it was the right choice for the initial version [of Facebook Chat] because of the limited resources we had at the time. It was probably also the right choice for WhatsApp for the same reason." So there's your answer for the situations in which to prefer using Erlang over Python, although I wonder if it's still the case now that Python has better(?) async support(?).
- The "open connections" Adam's talking about seems to be referring to WebSocket connections, which are used when you want the server to be able to 'push' information to the client immediately.
- ChatGPT: WebSockets are ideal for applications that require live updates, such as chat applications, collaborative editing tools, stock tickers, real-time dashboards, multiplayer games, and any other application where immediate data transmission is necessary.
- The 'collaborative editing tools' and 'multiplayer games' examples ChatGPT gives really stick out at me as clear examples of when you really need data transferred ASAP to avoid conflicts that will need to be resolved (like two people editing something in a Google Doc at the same time, or shooting at a person in a multiplayer game and then finding out that actually he had moved somewhere else).
- It seems like the apps that are using Erlang could also be built with Python, but using Erlang could require much fewer servers(?) and therefore require less complicated setups at the server level(?). To what extent, I'm not sure. But it doesn't seem like there are user features that these apps can achieve with Erlang that simply can't be done with Python.
What others say
- Adam D'Angelo - Why was Erlang chosen for use in Facebook chat?
While I was in college I made a prototype of a chat website. I wrote the long polling server in Erlang because it seemed like the ideal language for the task and also as a chance to have fun learning a new language.
I then worked on the initial Facebook chat prototype as part of a hackathon project in early 2007, along with Rebekah Cox and Ari Steinberg. I reused some of the Erlang code from my personal project. When we decided to make it an official project and productionalize the server code, the team decided to stick with Erlang and took over the code from there.
The rationale was something like:
- We already have a working prototype to start from.
- Erlang is good at holding a ton of open connections, which is the hardest part of a chat service.
- Erlang is easy to learn, even though very few engineers already know it.
The last point, lack of expertise, is a real weakness of the language, but it was outweighed by other factors at the time. Ultimately it contributed to Facebook's decision to move the service to C++, though even with the knowledge that it would eventually be rewritten, I think it was the right choice for the initial version because of the limited resources we had at the time. It was probably also the right choice for WhatsApp for the same reason.
See Ben Maurer's answer to When did Facebook switch away from using Erlang for Facebook Chat? for more details about why Facebook later switched.
- Ben Maurer - When did Facebook switch away from using Erlang for Facebook Chat? What was the reason for it?
At the time Facebook switched away from Erlang, the chat system had numerous reliability issues. The Erlang servers (which were used to provide long-poll functionality for browsers) caused some, but certainly not all of the problems. One problem we found with Erlang was that some of the abstractions Erlang had that allowed one to transparently have a distributed system across multiple machines actually caused reliability problems -- one server in the group would fail and cause cascading issues. Secondly, we found a number of instances where Erlang wasn't scaling well to multiple cores.
I don't think these issues are necessarily fundamental flaws of the Erlang runtime. It's quite possible that there was tuning that would have made Erlang work better for us (in fact, in our investigations we found a few tuning options that helped us).
However, Facebook has a number of useful abstractions in C++, making that a logical replacement. The server we ended up writing uses the same HTTP server that we use to balance all incoming requests to Facebook's website. This server is actively maintained -- for example, we spend time adding SPDY support and making SSL faster. Our open source Folly library provides great C++ abstractions for writing servers. We're able to reuse our high performance Thrift library.
In the end, I believe being able to reuse internal abstractions provided a substantial advantage. Many of these abstractions weren't available (or were in much more primitive states) when the initial Erlang service was written. Thus I believe Erlang was likely the logical choice when the chat service was first written (but I was not around then, so can't speak as to the merits of the choice at the time)
- 2011 - StackExchange - Software Engineering - Erlang or 'x'.. Is it worth it (vs. Python w/ GIL)?
- Question: I recently learned about CPython's GIL (Global Interpreter Lock), which raised my eyebrow a bit. If I understand it correctly, this means that if I have a Queue in my web app and have threads that complete jobs in the queue, then the code is locked until the thread for that request is complete, meaning that any subsequent request is locked while the previous thread is executing. Has anyone in "real world" applications found this to be a problem? Is it worthwhile to learn a language that supports real concurrency out of the box, such as Erlang? I'm most interested in any benchmarks that anyone has done in real world apps and whether or not anyone has seen any real noticeable issues with the GIL.
- Answers:
2011 - I have run up against the GIL in server side programming in almost every instance where I need something to scale to millions of concurrent users on multiple core machines. Python is great for command line tools and things that don't need true concurrency to extract every last bit of performance from a given piece of hardware. But for things that really need to squeeze everything out of something like a Sun T2000, you don't want to write anything in Python, it will be a operational maintenance nightmare running 32 separate processes and trying to management them all. I abandoned Twisted in favor of Erlang a few years ago, Python just doesn't cut it in the large scale concurrency space. The transparent distributed nature of Erlang means it scales horizontally as well as vertically.
- My thoughts:
- I wonder if the issues that these devs point out are still issues, given all the new features Python has come out with.
- 2012.10.20 - Evan Miller - Why I Program in Erlang
Apps using it
- AdRoll
- Goldman Sachs HFT (source)
- "Erlang is used as part of the real-time monitoring solution for this distributed trading system."
- Pinterest (source)
- "routes more than 30K events per-second to its in-house rule engine for preventing spam"
- "uses Elixir to power its rate limit service with sub-millisecond response times in its API endpoints"
- WhatsApp
- WhatsApp wasn't started with vanilla Erlang but rather with ejabberd, and they then had to rewrite parts of it with custom Erlang to improve its performance.
Articles
- Quora - Which is better, Python or Erlang?
- 2011.08.05 - Mixpanel - How and Why We Switched from Erlang to Python
- 2014? - FastCompany - Inside Erlang, the rare programming language behind WhatsApp's success
Related websites
- 2013.10.28 - Stack Overflow - What is the technology behind wechat, whatsapp and other messenger apps?
- Good info here.
How to learn it
Books
- Programming Erlang: Software for a Concurrent World
- Described by many of the reviews as the best book out there on Erlang.
Tutorials
- Compilations of tutorials
- Medium - Become an Erlang Cowboy and tame the Wild Wild Web — Part I
- Evan Miller - The Joy of Erlang
- Evan Miller - Chicago Boss - Tutorial
We love Rails, but we believe that Ruby is the wrong choice of platform on which to build a simple, fast, reliable website. The alternatives aren't much better: the JVM, V8, Python, and Perl all eventually run into the problem of dog-slow server-side templates.
Web developers usually "solve" the problem in a number of ways: buying lots of hardware, hiring a big ops team, implementing complex nested hierarchical cache structures, ripping out features, and moving to a "thick-client" architecture that takes forever to load. But there's a much simpler solution: it's called respecting the RAM.
Erlang Respects Your RAM!
Erlang is different from other platforms because when rendering a server-side template, it doesn't create a separate copy of a web page in memory for each connected client. Instead, it constructs pointers to the same pieces of immutable memory across multiple requests.
So if two people request two different profile pages at the same time, they're actually sent the same chunks of memory for the header, footer, and other shared template snippets. The result is a server that can construct complex, uncached web pages for hundreds of users per second without breaking a sweat.
With Erlang, you can run a website on a fraction of the hardware that Ruby and the JVM require, saving you money and operational headaches. You can run a website on "wimpy-core" ARM servers, helping your bottom-line along with the environment. Your test suite will run much faster than you thought possible, shortening your development cycle and letting you deliver features more quickly.
Chicago Boss Makes Erlang Accessible
Erlang has a reputation for being a mystical technology available only to an elite few willing to spend years learning its arcane syntax. It's true that Erlang "looks weird" to developers accustomed to C-like syntax, and the semantics of functional programming takes some getting used to. By adopting familiar Rails conventions, Chicago Boss has finally made Erlang accessible to programmers who don't have years and years of free time.
- TryErlang.org
- Introduction to Erlang
- Integers
- Integers can be expressed in a base other than 10. In this case, the Base#Value notation is used.
- In Erlang any expression in the shell must be terminated by a dot (.).
- Expressions using arithmetical operators
- The / operator is used for floating point division while the div operator is used for integer division.
- Atoms
- Atoms are constant literals that stand for themselves. They are similar to enumeration types in other programming languages.
- Atoms start with a lowercase letter or are delimited by single quotes.
- If the atom is not enclosed in quotes, valid characters are letters, digits, the AT symbol (@) and the underscore (_).
- If the atom is enclosed in single quotes, any character is allowed.
- Booleans
There is no separate Boolean type in Erlang. The atoms true and false are used instead. The following Boolean operators are available: and, andalso, or, orelse, xor, not
- andalso and orelse operators are shortcut evaluations of and and or. In other words, they will not evaluate the second argument when this is not needed to determine the final result of the Boolean expression.
- NW: Why don't the 'and' and 'or' statements behave like that by default?
Expressions
- Expressions can be compared using the following operators:
- == Equal to (not paying attention to the type, e.g. 1 == 1.0)
- /= Not equal to (not paying attention to the type)
- =:= Exactly equal to
- =/= Exactly not equal to
- =< Less than or equal to
- < Less than
- >= Greater than or equal to
- > Greater than
- Expressions can be compared using the following operators:
- Variables
- There's a huge difference between Erlang and most of the other conventional programming languages. In Erlang, once you've bound a variable, you cannot change its value. If you need to do a computation and manipulate the value of a variable you need to store the results in a new variable. This is often referred as "Single Assignment".
- Integers
- The Erlang Shell
- Lists and Tuples
- Pattern Match
- Functions
- List Comprehensions
- Introduction to Erlang
- 2007 - Thinking in Erlang (pdf)
- This looks pretty good.
- 2008.05.01 - Jesse Farmer - Learning Erlang
- Summary
- Variables vs. Atoms
- Variables in Erlang are very different from most other languages: they're either bound or unbound, and bound variables cannot be rebound in the same context. This means that variables are write-once.
- "context" in Erlang means lexical scope. There is no global scope.
- Variables in Erlang always start with a capital letter.
- var is treated as an atom by Erlang. Atoms satisfy the same role that symbols do in Ruby. Any literal that isn't another data-type, variable, or function is an atom.
- Atoms usually start with lower-case letters but you can also denote atoms by enclosing the name in single quotes.
- Data Types
- Erlang has all the "favorite" data types: integers, floats, strings, etc. It also has Funs, or "functional objects," which are anonymous functions.
- Erlang also has two basic compound data types: lists and tuples.
- There are no booleans in Erlang. Instead the atoms true and false are used.
- Assignment vs. Pattern Matching
- Erlang matches patterns and variables will match any pattern.
- Example: in the statement {X, Y} = {first, {second, third}}, X will be set to 'first', and Y will be set to {second, third}.
- NW: Doesn't seem so different from Python's multiple-variable assignment statements.
- Erlang matches patterns and variables will match any pattern.
- Looping vs. Recursion
- Since variables are bound to their lexical scope it makes procedural-style looping in Erlang difficult.
- Instead loops are done through recursion.
- Tail Recursion
- Since iterative loops are difficult in Erlang making sure your recursive functions are tail recursive is important. This means the last call a recursive function should make it to itself.
- NW: I don't really understand this.
- To pass data you can use accumulators.
- Since iterative loops are difficult in Erlang making sure your recursive functions are tail recursive is important. This means the last call a recursive function should make it to itself.
- Variables vs. Atoms
- Summary
Elixir
- Elixir
- Rec'd by Fabian Merkle, he said it makes Erlang easier to use.