Using wget to crawl your website

April 2, 2018

If you are looking to crawl your website for something like cache warming you can do so with wget quite easily. The following wget command will crawl a site and leave nothing behind on the local filesystem afterward. wget –mirror -q -e robots=off -p -r –delete-after -nd –mirror crawls the entire site -q prevents

Quickly list all hosts in your ssh config

April 4, 2017

I try to be a good citizen and create unique ssh keys for each service I ssh into. I setup all sorts of fun config items and one of my favorites is host aliases. I have accumulated dozens of hosts in my config file and remembering the names of each can be problematic. I tend to

Bash script to automatically convert git submodules to regular files

August 3, 2016

Git submodules drive me batty! They are a great idea in theory however in practical application they are a pain in the butt to work with. I have a project that has accumulated over a dozen submodules over the past couple years. Switching branches and merging anything has become excruciating. This morning was the last straw.

Find largest files in a directory using the terminal

June 1, 2016

If you need to find out what the largest files are in a directory and subdirectories here is a command you can run that will find the top 5 largest files. find . -type f | sed ‘s/.*/”&”/’ | xargs ls -Slh | head -n 5 Breakdown: find . -type f: Find all files in

Autocomplete git branches and commands in bash

February 10, 2016

The tab completion for commands in bash is awesome, but it does not work for bash out of the box. Lucky for us it is relatively easy to get bash completion working. You will need two things, a copy of the completion script and an update to your bash profile. The completion script is available