AWS

EC2

• 2012.06.13 - EC2 for Poets

Apache (HTTP server)

How Paul set up the web.py server with Apache on Digital Ocean

1. Remember that the default Apache config file is located at /etc/apache2/apache2.conf, but Paul said you generally don't need to make changes to it.

2. Instead, because we're using a Ubuntu instance (and Ubuntu is based on Debian), the note at the top of the apache2.conf relates to us when it says that the configuration can be broken into separate files, and that in particular, we can use the option that says IncludeOptional sites-enabled/*.conf

3. So what he did was to create a /etc/apache2/sites-available/oorklecom.conf file and a /etc/apache2/sites-available/ioorklecom.conf file, and then create symlinks to those files in the /sites-enabled/ folder that had the same names as the original files.

4. The /sites-available/ folder had a file named 000-default.conf which had these contents:

   1. 
      
      

      <VirtualHost *:80> # The ServerName directive sets the request scheme, hostname and port that # the server uses to identify itself. This is used when creating # redirection URLs. In the context of virtual hosts, the ServerName # specifies what hostname must appear in the request's Host: header to # match this virtual host. For the default virtual host (this file) this # value is not decisive as it is used as a last resort host regardless. # However, you must set it for any further virtual host explicitly. #ServerName www.example.com ServerAdmin webmaster@localhost DocumentRoot /var/www/html # Available loglevels: trace8, ..., trace1, debug, info, notice, warn, # error, crit, alert, emerg. # It is also possible to configure the loglevel for particular # modules, e.g. #LogLevel info ssl:warn ErrorLog ${APACHE_LOG_DIR}/error.log CustomLog ${APACHE_LOG_DIR}/access.log combined # For most configuration files from conf-available/, which are # enabled or disabled at a global level, it is possible to # include a line for only one particular virtual host. For example the # following line enables the CGI configuration for this host only # after it has been globally disabled with "a2disconf". #Include conf-available/serve-cgi-bin.conf </VirtualHost> # vim: syntax=apache ts=4 sw=4 sts=4 sr noet

      
      
      

5. He set up the oorklecom.conf file to look like this:

   1. 
      
      

      <VirtualHost *:80> # The ServerName directive sets the request scheme, hostname and port that # the server uses to identify itself. This is used when creating # redirection URLs. In the context of virtual hosts, the ServerName # specifies what hostname must appear in the request's Host: header to # match this virtual host. For the default virtual host (this file) this # value is not decisive as it is used as a last resort host regardless. # However, you must set it for any further virtual host explicitly. ServerName www.oorkle.com ServerAlias oorkle.com ServerAdmin webmaster@localhost DocumentRoot /var/www/oorkle WSGIDaemonProcess oorklemobile threads=5 WSGIScriptAlias / /var/www/oorkle/oorkle.py Alias /static /var/www/oorkle/static/ # Available loglevels: trace8, ..., trace1, debug, info, notice, warn, # error, crit, alert, emerg. # It is also possible to configure the loglevel for particular # modules, e.g. #LogLevel info ssl:warn ErrorLog ${APACHE_LOG_DIR}/error.log CustomLog ${APACHE_LOG_DIR}/access.log combined # For most configuration files from conf-available/, which are # enabled or disabled at a global level, it is possible to # include a line for only one particular virtual host. For example the # following line enables the CGI configuration for this host only # after it has been globally disabled with "a2disconf". #Include conf-available/serve-cgi-bin.conf </VirtualHost> # vim: syntax=apache ts=4 sw=4 sts=4 sr noet

      
      
      

6. The Web.py WSGI deployment docs mention the WSGIScriptAlias command, and he said that he'd already known about the WSGIDaemonProcess command from having worked with Flask.

   1. More info on WSGIDaemonProcess here.

7. At that point I assume you can just restart the Apache server with /etc/init.d/apache2 restart (Source)

Docker

Intros

• ROpenSciLabs.github.io - What is Docker and Why should I use it?

  • This first page of the tutorial was a good quick overview of what Docker is for:

  • In short, you should use Docker because

    • it allows you to wrangle dependencies starting from the operating system up to details such as R and Latex package versions

    • it makes sure that your analyses are reproducible.

    There are a couple of other points what Docker helps with:

    • Portability: Since a Docker container can easily be sent to another machine, you can set up everything on your own computer and then run the analyses on e.g. a more powerful machine.

    • Sharability: You can send the Docker container to anyone (who knows how to work with Docker).

    Basic vocabulary

    The words image and container will come up a lot in the following. An instance of an image is called container. An image is the setup of the virtual computer. If you run this image, you will have an instance of it, which we call containter. You can have many running containers of the same image.

• Docker - Get Started

  • The difference between a container and a virtual machine:

    • A container runs natively on Linux and shares the kernel of the host machine with other containers. It runs a discrete process, taking no more memory than any other executable, making it lightweight.

      By contrast, a virtual machine (VM) runs a full-blown “guest” operating system with virtual access to host resources through a hypervisor. In general, VMs provide an environment with more resources than most applications need.

Important commands

• docker container ls

• docker <container_id> exec <command>

• docker build -t <desired image name> .

• docker run <image name>

• docker-machine restart

With Python

• Runnable.com - Dockerize your Python Application

Troubleshooting / debugging

• 2016.03.23 - Medium - Ten tips for debugging Docker containers

HTTPS

Let's Encrypt

• https://letsencrypt.org/

  • rec'd by Naveed

Map / Reduce

• A brief explanation of what map / reduce does: 

NGINX

Tools

• https://nginxconfig.io/

Articles

• ServerDensity - Troubleshoot Nginx: 10 typical errors

• Nginx.com - Serving static content

• Nginx.org - Beginner's guide

Misc thoughts

• 2017.03.08 - While troubleshooting nginx for Service Fusion, the config files I needed to look at were in /etc/nginx/. I just searched all of the files at once with a single command to see which of them had the text string I was looking for. It turned out to be in /etc/nginx/sites-enabled/default.

SSH

Pageant

• Making Pageant automatically load keys on startup

Task / Job queues

Celery

• a quick youtube intro
  
  

  • rec'd by Cody

Gearman

• rec'd by Mike Krieger at Instagram

  • We use gearman (http://www.gearman.org/) for this task. There's a pretty good Python library to interface with it. Much easier to set-up than Celery / RabbitMQ, in my experience.

Time-tracking

Chronolapse

• https://www.chronolapse.com/

• Use Chronolapse to record your screen.

  • My settings for my dual-monitor set-up:

    • Top: 1919

    • Left: -550

    • Width: 1925

    • Height: 1050

• I couldn't get Chronolapse to convert the screens to a gif / video (it produced an error), so I used: http://gifmaker.me/

  • I set it to create a gif at 50% of the size.

  • If you do 1 capture per minute, you end up with a ~12mb/hour file.

tmux

tmux lets you access multiple terminal sessions simultaneously in a single window.

Rec'd by

• Cody

Tutorials

• https://www.sitepoint.com/tmux-a-simple-start/

  • rec'd by someone as a good guide

• Cygwin tmux packages

Vim

• Vim for Humans (ebook)

Tutorials / training

• I asked for advice to find a 3-minute Vim command training circuit here.

  • Motions in normal mode:

    • hjkl for one-off movement. Use sparingly.

    • Word-wise motions. w [g]e b and their uppercase cousins W [g]E B.

    • Learn to use the Numbers. 5w jumps five words ahead.

    • / to find stuff. n to jump to the next (partial) hit. N to jump to the previous one.

    • * and # to jump to the next/previous occurence of the word under the cursor.

  • Edits:

    • Insert at beginning, Append at the end of line, ciw Change in word d]} delete until next sentence, or paragraph Substitute line with your Input...

    • . repeat last action

    • undo and <C-r> redo

  • Ex-Commands:

    • :s/find/replace find the word "find" and replace it with the word "replace", in the current Line

    • :%s//newword/gi - replaces the last search (with /) with "newword" for the whole file. The /gi - options make it that you replace all occurences (global) and that the replacements are case-insensitive.

    • :w write/save changes

    • :wq! the meme. Save and quit. Also ZZ.

    • q: what have I done? Ex-command History Mode. :q to leave again.

    • :buffers :registers :marks to See what you edit, what you yanked, and where you Markes stuff. (Also searches and last Ex-Commands)

  • Visual Mode:

    • v V and <C-V>

    • in visual Mode: other end of selection

  • There are too many things to list here. See :helphelp.

• Apparently the official docs are good: “nothing, absolutely nothing, beats the documentation itself”

• :vimtutor is the official tutorial

• https://vim-adventures.com/

• https://www.theviminator.com/

• https://github.com/Weyaaron/nvim-training

• “Just play nethack.” (source) (Apparently it teaches you the movement keys hjkl.)

  • “hjkl are almost worthless compared to other means of navigation” (source)

• • “As a long time Vim user, I also found the book Practical Vim very instructive. Not really suitable for beginners though.” (source)

Vimium

Vimium is a Chrome extension that lets you control Google Chrome with vim-like keyboard commands.  It was used by some devs at Infer.

Documentation

• https://github.com/philc/vimium/wiki

How to learn to use Vimium

1. Memorize the Shift+/ command to open the 'Help' dialog.

2. Memorize the "f" to select an option on the page.

   • You could have a website where there are a bunch of links that the user needs to select by using this command.

3. Memorize "b" to open a bookmark.