Nathan Wailes - Blog - GitHub - LinkedIn - Patreon - Reddit - Stack Overflow - Twitter - YouTube
Tools (Programming)
- 1 AWS
- 1.1 EC2
- 2 Apache (HTTP server)
- 3 Docker
- 3.1 Intros
- 3.2 Important commands
- 3.3 With Python
- 3.4 Troubleshooting / debugging
- 4 HTTPS
- 4.1 Let's Encrypt
- 5 Map / Reduce
- 6 NGINX
- 6.1 Tools
- 6.2 Articles
- 6.3 Misc thoughts
- 7 SSH
- 7.1 Pageant
- 8 Task / Job queues
- 9 Time-tracking
- 9.1 Chronolapse
- 10 tmux
- 11 Vim
- 11.1 Tutorials / training
- 12 Vimium
- 12.1 Documentation
- 12.2 How to learn to use Vimium
AWS
EC2
Apache (HTTP server)
How Paul set up the web.py server with Apache on Digital Ocean
Remember that the default Apache config file is located at /etc/apache2/apache2.conf, but Paul said you generally don't need to make changes to it.
Instead, because we're using a Ubuntu instance (and Ubuntu is based on Debian), the note at the top of the apache2.conf relates to us when it says that the configuration can be broken into separate files, and that in particular, we can use the option that says
IncludeOptional sites-enabled/*.conf
So what he did was to create a
/etc/apache2/sites-available/oorklecom.conf
file and a/etc/apache2/sites-available/ioorklecom.conf
file, and then create symlinks to those files in the/sites-enabled/
folder that had the same names as the original files.The
/sites-available/
folder had a file named000-default.conf
which had these contents:<VirtualHost *:80> # The ServerName directive sets the request scheme, hostname and port that # the server uses to identify itself. This is used when creating # redirection URLs. In the context of virtual hosts, the ServerName # specifies what hostname must appear in the request's Host: header to # match this virtual host. For the default virtual host (this file) this # value is not decisive as it is used as a last resort host regardless. # However, you must set it for any further virtual host explicitly. #ServerName www.example.com ServerAdmin webmaster@localhost DocumentRoot /var/www/html # Available loglevels: trace8, ..., trace1, debug, info, notice, warn, # error, crit, alert, emerg. # It is also possible to configure the loglevel for particular # modules, e.g. #LogLevel info ssl:warn ErrorLog ${APACHE_LOG_DIR}/error.log CustomLog ${APACHE_LOG_DIR}/access.log combined # For most configuration files from conf-available/, which are # enabled or disabled at a global level, it is possible to # include a line for only one particular virtual host. For example the # following line enables the CGI configuration for this host only # after it has been globally disabled with "a2disconf". #Include conf-available/serve-cgi-bin.conf </VirtualHost> # vim: syntax=apache ts=4 sw=4 sts=4 sr noet
He set up the oorklecom.conf file to look like this:
<VirtualHost *:80> # The ServerName directive sets the request scheme, hostname and port that # the server uses to identify itself. This is used when creating # redirection URLs. In the context of virtual hosts, the ServerName # specifies what hostname must appear in the request's Host: header to # match this virtual host. For the default virtual host (this file) this # value is not decisive as it is used as a last resort host regardless. # However, you must set it for any further virtual host explicitly. ServerName www.oorkle.com ServerAlias oorkle.com ServerAdmin webmaster@localhost DocumentRoot /var/www/oorkle WSGIDaemonProcess oorklemobile threads=5 WSGIScriptAlias / /var/www/oorkle/oorkle.py Alias /static /var/www/oorkle/static/ # Available loglevels: trace8, ..., trace1, debug, info, notice, warn, # error, crit, alert, emerg. # It is also possible to configure the loglevel for particular # modules, e.g. #LogLevel info ssl:warn ErrorLog ${APACHE_LOG_DIR}/error.log CustomLog ${APACHE_LOG_DIR}/access.log combined # For most configuration files from conf-available/, which are # enabled or disabled at a global level, it is possible to # include a line for only one particular virtual host. For example the # following line enables the CGI configuration for this host only # after it has been globally disabled with "a2disconf". #Include conf-available/serve-cgi-bin.conf </VirtualHost> # vim: syntax=apache ts=4 sw=4 sts=4 sr noet
The Web.py WSGI deployment docs mention the WSGIScriptAlias command, and he said that he'd already known about the WSGIDaemonProcess command from having worked with Flask.
More info on WSGIDaemonProcess here.
At that point I assume you can just restart the Apache server with
/etc/init.d/apache2 restart
(Source)
Docker
Intros
ROpenSciLabs.github.io - What is Docker and Why should I use it?
This first page of the tutorial was a good quick overview of what Docker is for:
In short, you should use Docker because
it allows you to wrangle dependencies starting from the operating system up to details such as R and Latex package versions
it makes sure that your analyses are reproducible.
There are a couple of other points what Docker helps with:
Portability: Since a Docker container can easily be sent to another machine, you can set up everything on your own computer and then run the analyses on e.g. a more powerful machine.
Sharability: You can send the Docker container to anyone (who knows how to work with Docker).
Basic vocabulary
The words image and container will come up a lot in the following. An instance of an image is called container. An image is the setup of the virtual computer. If you run this image, you will have an instance of it, which we call containter. You can have many running containers of the same image.
The difference between a container and a virtual machine:
A container runs natively on Linux and shares the kernel of the host machine with other containers. It runs a discrete process, taking no more memory than any other executable, making it lightweight.
By contrast, a virtual machine (VM) runs a full-blown “guest” operating system with virtual access to host resources through a hypervisor. In general, VMs provide an environment with more resources than most applications need.
Important commands
docker container ls
docker <container_id> exec <command>
docker build -t <desired image name> .
docker run <image name>
docker-machine restart
With Python
Troubleshooting / debugging
HTTPS
Let's Encrypt
rec'd by Naveed
Map / Reduce
A brief explanation of what map / reduce does:
NGINX
Tools
Articles
Misc thoughts
2017.03.08 - While troubleshooting nginx for Service Fusion, the config files I needed to look at were in /etc/nginx/. I just searched all of the files at once with a single command to see which of them had the text string I was looking for. It turned out to be in /etc/nginx/sites-enabled/default.
SSH
Pageant
Task / Job queues
Celery
rec'd by Cody
Gearman
rec'd by Mike Krieger at Instagram
We use gearman (http://www.gearman.org/) for this task. There's a pretty good Python library to interface with it. Much easier to set-up than Celery / RabbitMQ, in my experience.
Time-tracking
Chronolapse
Use Chronolapse to record your screen.
My settings for my dual-monitor set-up:
Top: 1919
Left: -550
Width: 1925
Height: 1050
I couldn't get Chronolapse to convert the screens to a gif / video (it produced an error), so I used: http://gifmaker.me/
I set it to create a gif at 50% of the size.
If you do 1 capture per minute, you end up with a ~12mb/hour file.
tmux
tmux lets you access multiple terminal sessions simultaneously in a single window.
Rec'd by
Cody
Tutorials
https://www.sitepoint.com/tmux-a-simple-start/
rec'd by someone as a good guide
Vim
Vim for Humans (ebook)
Tutorials / training
I asked for advice to find a 3-minute Vim command training circuit here.
Motions in normal mode:
hjkl
for one-off movement. Use sparingly.Word-wise motions.
w
[g]e
b
and their uppercase cousinsW
[g]E
B
.Learn to use the Numbers.
5w
jumps five words ahead./
to find stuff.n
to jump to the next (partial) hit.N
to jump to the previous one.*
and#
to jump to the next/previous occurence of the word under the cursor.
Edits:
I
nsert at beginning,A
ppend at the end of line,ciw
Change in wordd]}
delete until next sentence, or paragraphS
ubstitute line with your Input....
repeat last actionu
ndo and<C-r>
redo
Ex-Commands:
:s/find/replace
find the word "find" and replace it with the word "replace", in the current Line:%s//newword/gi
- replaces the last search (with/
) with "newword" for the whole file. The /gi - options make it that you replace all occurences (g
lobal) and that the replacements are case-i
nsensitive.:w
write/save changes:wq!
the meme. Save and quit. AlsoZZ
.q:
what have I done? Ex-command History Mode.:q
to leave again.:buffers
:registers
:marks
to See what you edit, what you yanked, and where you Markes stuff. (Also searches and last Ex-Commands)
Visual Mode:
v
V
and<C-V>
in visual Mode:
o
ther end of selection
There are too many things to list here. See
:helphelp
.
Apparently the official docs are good: “nothing, absolutely nothing, beats the documentation itself”
:vimtutor
is the official tutorialGitHub - Weyaaron/nvim-training: A plugin to practice neovim movements, currently in "open beta"
“Just play nethack.” (source) (Apparently it teaches you the movement keys hjkl.)
“hjkl are almost worthless compared to other means of navigation” (source)
Practical Vim: Edit Text at the Speed of Thought
“As a long time Vim user, I also found the book Practical Vim very instructive. Not really suitable for beginners though.” (source)
Vimium
Vimium is a Chrome extension that lets you control Google Chrome with vim-like keyboard commands. It was used by some devs at Infer.
Documentation
How to learn to use Vimium
Memorize the Shift+/ command to open the 'Help' dialog.
Memorize the "f" to select an option on the page.
You could have a website where there are a bunch of links that the user needs to select by using this command.
Memorize "b" to open a bookmark.