Nathan Wailes - Blog - GitHub - LinkedIn - Patreon - Reddit - Stack Overflow - Twitter - YouTube

Non-language/platform-specific topics (Programming)

Owned by Nathan Wailes
Last updated: May 29, 2023
12 min read

Table of contents

Child pages

Books

Articles


Bots

Twitter bots

Python Twitter bots

How to create a Twitter bot

  1. Create a new Twitter account.
    1. Go to twitter.com. 
    2. Click 'Sign up'. 
    3. Fill out the sign-up information and click 'Next'. 
      1. If you already have a Twitter account registered under your primary email address, you may need to create a new email address to register a new account.
    4. If it prompts you for your phone number, be sure to fill it out, as creating a bot requires that you register your phone number.

Clean coding

Cleaning up other people's code

  • Committed code shouldn't have extraneous stuff in it.
    • If a variable is never used and there's no explanation for why, it's probably an oversight. Feel free to delete it.

Books:
The Elements of Style by Strunk and White - rec'd by at least one of the guys in Coders At Work
Clean Code by Robert Martin - rec'd in comments here
Code Complete by Steve McConnell - rec'd by Allston, also rec'd in comments here
Refactoring by Martin Fowler et al. - rec'd by Allston

Websites:
http://twitter.github.com/bootstrap/
Google's Style Guides for Various Programming Languages - !!
7 Ways to Write Beautiful Code
Code Conventions for the Java Programming Language by Oracle

Articles:
Kotaku - The Exceptional Beauty of Doom 3's Source Code

Quote:
I was really excited to write this article, because it gave me an excuse to really think about what beautiful code is. I still don’t think I know, and maybe it’s entirely subjective. I do think the two biggest things, for me at least, are stylistic indenting and maximum const-ness.

Books

Clean Code

  • I read this book cover-to-cover twice and it really helped me to improve the quality of my code, I've gotten compliments from people since then about how easy my code is to understand.
  • The basic idea is that your code should read like plain English as much as possible.
  • Summary: https://gist.github.com/cedrickchee/55ecfbaac643bf0c24da6874bf4feb08

Compilers / Language Design

Background

  • Compilers are basically the way you build new programming languages on top of the old ones.
  • So, for example, Python was built with C.
  • My understanding is that you could theoretically use these principles to design a programming language that would take everyday English and turn it into a computer program. Obviously you would have a lot of issues with the ambiguity of everyday English.

Articles / Videos

Books

  • Books that looks like they're easy-to-read:
    • Compiler Design in C
      • I bought this book after reading the Dragon Book (Aho,Sethi,Ullman). While Dragon Book is more on theory and less on practical compiler design, this book will take you to the very foundation of compilers both theory and practical in a very simple and easy to understand language and with lots of examples.
    • Introduction to Compiler Construction
      • I'm a computer science professor and have taught compiler construction for several years. Parsons teaches the material the way it ought to be. He is clear and concise without leaving out any of the essential material.

      • I think Parsons had an intelligent idea in writing this book. Without originality claims (as he point out in the preface) the book is intended to the audience of novices, with the clear target of explaining in great details compilers principles. As he says in the preface, the objective is to prepare the reader for more advanced lectures, and he explicitly mention the reference book as an example: Aho-Sethi-Ullman's "Compilers: Principles, Techniques and Tools", a.k.a the Dragon book.
        The approach is quite theoretical and principles-centered, just as the Dragon book is. But Parsons departs from this in the writing style: It is definitely straightforward. He sacrifices the scope of the book in favor of clarity: he took the core of books like the Dragon, and expanded this core to a well appreaciable extent. It comes over and over again on more awkward concepts with detailed examples.

    • Language Implementation Patterns: Create Your Own Domain-Specific and General Programming Languages
      • 4.5 stars, 26 reviews
      • Guido van Rossum, creator of Python (Python forever!), commented: Throw away your compiler theory book! So I knew. I also found out that professor Parr has been teaching language applications programming for years. Then I knew. The book itself came from the famous Pragmatic Bookshelf. And I knew: LIP would be a good read.
  • Other books
    • Compilers: Principles, Techniques, and Tools
      • 4 stars, 97 reviews
      • I spent some serious quality time with the first edition (the "red dragon book"), in three main episodes over the past dozen years: 1) undergraduate compilers class, 2) industry project, and 3) parser generator implementation. During all three episodes, I was disappointed in various ways, though there is no denying that the book contains a wealth of information. As an undergraduate, I found the book somewhat impenetrable. When in industry, I found the book too abstract. When implementing a parser generator, I discovered that the book excludes important research results with regard to LR parser generation.
    • Principles of Compiler Design
      • 5 stars, 9 reviews
    • Engineering a Compiler
      • 4.5 stars, 12 reviews
    • Domain-Specific Languages
      • 4 stars, 14 reviews
      • Written by Martin Fowler, who also wrote "Refactoring"

      • As usual, Fowler delivers a very well structured book, easy to both read and use as reference material. He is a very able and pragmatic writer and that shows in this book.

        However, I can't consider this book a good text because of the things it omits. This is a book about designing DSLs and this task is one of the things functional languages excel at, but Fowler establishes in the introduction that he is going to happily ignore all things related to functional programming and never looks back. Anyone interested in designing DSLs owes it to himself to research Haskell, Scala and F# as they are vastly superior to Java in this respect.

Design Patterns

Dependency Injection

From Yang:

I recommend following dependency injection rules, one of which is that you should not consume "dependencies" like a database from within a constructor—constructors should always be dead simple and generally only initialize variables to values that are passed in (perhaps with some pure computations as is done here).

DI might be a little bit complicated so I can explain this to you in person some time over lunch or something, but it'll provide great reading and thought material. For now look over (from simplest exposure to most verbose):

http://stackoverflow.com/questions/130794/what-is-dependency-injection
http://www.jamesshore.com/Blog/Dependency-Injection-Demystified.html
http://tutorials.jenkov.com/dependency-injection/index.html
http://en.wikipedia.org/wiki/Dependency_injection
http://martinfowler.com/articles/injection.html
https://msdn.microsoft.com/en-us/magazine/cc163739.aspx

Factory Pattern

There are so many bad explanations out there...

Good explanations

DevOps / SRE / Sysadmin / Continuous Deployment

Books

Recommendations from Jacob from Infer:

  • "Most of the DevOps stuff I've been doing recently has been Postgres related. Postgres has great documentation so I might start there. One Postgres ecosystem tool we use is wal-e, which handles writing our Write Ahead Logs to S3. Speaking of S3, Amazon Web Services is another core platform piece we use. It also has documentation, although there's a lot going on. Specific AWS services we use include EC2, EBS, RDS, S3, SQS, and Route53. We interact with AWS mainly through a python library called boto. Finally, all of our systems are Linux, which is another entire rabbit hole -- I'm not sure what the right starting point is there."
  • https://eng.uber.com/mysql-migration/ - rec'd in Slack by Jacob

Ansible

Tutorials

Books

Functional Programming

tutorials

Good Explanations of General CS Topics


Hacking

  • Defcon 21 - Defense by numbers: Making Problems for Script Kiddies and Scanner Monkeys
    • Summary: Script kiddies use one of a few different known existing software applications that allow them to basically press a button to scan a server for vulnerabilities and then launch attacks. These software applications attempt to mimic existing web browsers (IE, Firefox, Chrome) when they send requests to web servers, and these browsers have certain quirks with how they handle uncommon HTTP response codes. Thus, if you are trying to defend your website against these kinds of low-sophistication script-kiddie attacks, you can use HTTP codes to cause problems with the attacking software to 1) dramatically increase the amount of time it takes for the software to find vulnerabilities (from minutes to 14+ hours), 2) reduce the number of vulnerabilities that end up being found, and 3) introduce false positives (fake vulnerabilities). The creators of these software applications coukd somewhat fix these current vulnerabilities by looking more closely at the responses they're getting from web servers, but 1) they don't feel any strong motivation to do it at the moment, and 2) this will cause some unavoidable slowdown in how the software works. He'd like to make it easy for web server admins to implement this kind of defense and so he'd like to get this defense included in an open-source project, but he doesn't have the skills necessary to do it, and so he asked if anyone in the audience would be interested.

IDEs

Sublime Text 2

IIRC Pieter Levels was using this for years, I don't know if he still is.


Object-oriented programming

Open Source

Projects I should consider contributing to

  • Anki



  • http://blog.smartbear.com/programming/1 ... rock-star/
  • http://stackoverflow.com/questions/4364 ... ce-project
  • http://stackoverflow.com/questions/2284 ... ce-project
  • Google Talk: How Open Source Projects Survive Poisonous People
    • http://www.youtube.com/watch?v=Q52kFL8zVoM
    • 2:00 - They highly recommend Karl Fogel's book "Producing Open-Source Software"
    • 2-3 - They give an overview of four-stages of protection:
      • 1) Comprehension
      • 2) Fortification
      • 3) Identification
      • 4) Disinfection
    • 3 min - They make the point that an open-source project's most valuable resource is the attention and focus of its community (contributing members), and that poisonous people will threaten that resource
    • 3-4 - Poisonous people can 1) distract your developers, 2) emotionally drain your community by squabbling or starting infighting. Some people do this on purpose, others do it accidentally.
    • 4min - OCD / perfectionists / people obsessed with progress can unintentionally derail progress [example: endless discussion]. Remember "the perfect can be the enemy of the good".
    • 5-6min - They tell the "painting the bikeshed" story, which has the MI that the amount of discussion that a group will engage in is inversely proportional to its complexity, because the simpler it is, the more people think they know enough to say something.
    • 7min - They start talking about fortifying.
    • There are four things they think make a healthy community:
      • 1) politeness
      • 2) respect
      • 3) trust
      • 4) humility
    • 8min - Have a direction and then limit your scope, so you don't end up with limitless feature creeping.
      • They give an example from their project "Subversion", where they had a big mission statement on their front page for years, so that anytime someone wanted to add a new feature, they could point to the mission and say "That's a great idea but it's outside our mission". Their statement was "To create a compelling replacement for CVS."
      • They recommend that if someone comes up with a feature, you say "Come back after we've finished the stuff on our to-do list".
      • They also enumerated at the beginning the exact features they wanted to do and what bugs in CVS they wanted to fix.
    • 10min - They give a second example, from the project "Google Web Toolkit". They started by coming up with a very specific mission statement that would strictly limit the scope of the project. "If you're trying to preserve attention and focus, the best way to do that is to have something to focus on."
    • 11min - The majority of official communication happens on mailing lists [I didn't know that]
      • Send new people to the archives to dig through old discussions before starting to talk, because if they don't do that they're disrespecting everyone else's time.
      • Don't respond to every message in a thread; it's annoying. Come up with one well-thought out post. And don't let people do this if you're running a project.
    • 13min - Document your project: 1) Design decisions, 2) bug fixes, and 3) mistakes (to avoid having new people repeat them), 4) code changes
    • 14min - They say that the usual way that open-source projects work is that someone will build up trust and get commit access, and then they'll just start committing changes and the other people in the project will just trust that the guy isn't messing things up. There aren't usually code reviews before each commit. 
      • They recommend having commit emails, and encouraging other members to review those email messages.
    • 15min - Don't let people work on some huge feature on their own and then commit it all when it's too big for anyone to understand how it works. You want everyone to understand how everything works.
    • 16min - Increase the "bus factor" of your code: the lowest number of people in your group who, if they were hit by a bus, could cripple your project.
    • 17-19 - They say that what often happens in corporations and other places is that people will mark off their territory and make other people feel unwelcome about dealing with that part. They recommend that you not allow people to put their names at the top of source-code files. It also introduces politics re: "How much do I need to contribute to get my name at the top of the code?"

Problem-solving / debugging / troubleshooting

Background

  • This is one of the biggest parts of being a programmer.
  • It is a very common weakness among weaker software engineers to simply fret when they hit a hard problem instead of knowing how to decompose it into smaller problems.
    • Source: C. had said this to me.


  • Get a link to the YouTube video by Tom Francis in which he describes having trouble debugging something. What I was struck by was his description of his debugging process: just disabling code until the bug disappeared, and then using that to figure out what was causing the bug. (In this case the bug was that the player's character wouldn't respond to controls on a particular level.)

Books

  • The Pragmatic Programmer
    • Chapter 18, 'Debugging'
    • Chapter 37, 'Solving Impossible Puzzles'

Articles

P vs. NP

2015.11.10 - ScienceMag - Mathematician claims breakthrough in complexity theory

  • This is a very well-written article.
  • I finished reading it with a good understanding of what P vs. NP is all about.

Reviewing other people's code ('Code Review')

Articles / Videos

Scalability

Search

Software Architecture

Specs







Follow me on Twitter