Archive for September, 2012
I’ve known about ssh-agent for a while, but as I was practically permanently using PuTTY (on Windows), I only bothered with learning about Pageant.
But Git uses ssh to connect to github, and I was getting tired of typing in my password with every push. I got annoyed with InteliJ for making me type in my password in with every push, and this was no different. Because git uses bash as its’ command line on both Windows and Linux, I decided to get started with using ssh-agent.
The first time I ran ssh-agent expecting it to work automagically. Instead it dumped what looked like two environment variables to the screen and quit. Not too helpful, but I manually copied, formatted & pasted the variables, and got it working.
But doing that manually, while awesome when pushing from one system and pulling on another, was also annoying with having to do it every time I logged in. So I looked into automating it. I knew of eval and backticks in bash. So I tried `ssh-agent`.
sh.exe": SSH_AUTH_SOCK=/tmp/ssh-myYvgp1404/agent.1404;: No such file or directory
Hmm. No joy. Ok, let’s try eval ssh-agent. Maybe that’ll make a difference?
$ eval ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-zIQZKN6080/agent.6080; export SSH_AUTH_SOCK;
SSH_AGENT_PID=1092; export SSH_AGENT_PID;
echo Agent pid 1092;
Nope, back to manually editing it. Hmm. Google search time.
Which led me to a uni helpdesk page. Which told me that I had to use both eval and backticks. Huh. How ingenious.
$ eval `ssh-agent`
Agent pid 4288
OH HEY THAT LOOKS GOOD.
So I gleefully ran it on the uni’s servers. And then discovered I had to run it in every bash shell. Hmm. Sounds kind of unoptimal. Fixed my original problem, which was having to type in my password each and every time I pushed or pulled from github. And using eval and backticks meant it was easier than copy & pasting the output. But I didn’t like having to run ssh-add and ssh-agent in every new session, or that ssh-agent wouldn’t auto-terminate when I closed bash. (Which led to a surprise when I ran
ps u and saw at least 8 ssh-agent processes running with my username.)
So I just ran it in a screen session. And just have to use that one window everytime. Fairly straightforward.
The alternative was add to the commands to my .bashrc. But the uni servers seem to do something to bash such that it doesn’t execute a user’s bashrc when they login, only after they run bash. So running it in screen works just as well, since I’d need to run a command anyway.
As an alternative alternative, I also found a script that should work for keeping track of ssh-agent and can be run from .bashrc, so I’ll be looking into that too.
Note: This is a personal VPN, so I just used static keys. A general guide to getting OpenVPN set up is available on the OpenVPN website, but this guide is targeted at CentOS 5 on an OpenVZ VPS.
This guide should be usable in other RH derivatives without much (any?) modification; and with slight modifications for debian-style distros, especially in installing packages and folder paths.
If you’re not running OpenVZ, I’d recommend following the site where the vast majority of this guide comes from, but I had problems with it – I had to mess around with the config files, and the iptables commands *will* kill your SSH session if you run it. Read the rest of this entry »
Note to self: When configuring iptables, don’t copy + paste
/sbin/iptables -F /sbin/iptables -P INPUT DROP <bunch of other commands>
I had A Bad Time.
So… I found a cheap VPS for me to play around with on LowEndBox after a few weeks of lurking on their site.
$12/year for 256MB ‘guaranteed’ RAM, 15GB of disk space and 300GB of bandwidth. I am pleased.
Except for the fact that it’s an CentOS OpenVZ instance, and I have had bad experiences with OpenVZ.
But other than being unable to run HLDS (runs out of RAM, gets killed by the host), nothing else has had problems. Got nginx, mysql & php-fpm on it following www.mellzamora.com/install-nginx-php-5-4-php-fpm-on-centos6/
I was pretty surprised though – First login showed only ~15MB of RAM used. WITH Apache running. (But no PHP or external modules.) But still. 15MB.
Now, as for uses of it… well, that’s yet to be established. I’ll probably be moving my IRC bot over from EC2, because the credit on that is running out end of October.
As for more configs:
www.digitalocean.com/community looks like it has a bunch of tutorials on getting nginx and the like set up
And github.com/KnightSwarm/Minstall has stuff on minimal installs.
I’ll be doing a fair amount of work in Python in the next few months, so I decided to sit down and get a good dev environment going. First on my laptop (32 bit is easier to deal with), then on my desktop.
So I’ll be doing 3 things:
- Getting Git setup
- Getting Git working with GitHub
- Getting Python and pip installed
First of all, started off with Git. Downloaded from git-scm.com/downloads, and installed it following GitHub’s Windows setup guide. (As a side note, I’m not dealing with GitHub’s native app because when I last used it, you couldn’t select individual blocks of code to be committed. I know Git philosophy is to commit often, but I just commit when I’ve got something working, and I’d have touched a few different files and done more than one thing.)
They’ve improved the installer since I last used it, so it was deceptively simple. No more messing around with config files, whee!
Second thing was integration with Github. SSH keys allow password-less authentication (I hate InteliJ’s Github integration because it uses the HTTPS repo, which requires me to enter my github password). Once again, easiest thing to do was to follow Github’s guide on generating ssh keys. Another side note: It’s ssh -T [email protected]. I was trying ssh -T github.com and was wondering why it was failing. =|
And onto the third and final thing, which is also the most difficult: Getting Python up and running with pip installed & working. I’ll split this into two parts: getting Python, and getting pip working.
Getting Python is trivial – I downloaded it from www.python.org/download/. I chose 2.7.3, but could have gone with 3.2.3. (In fact, probably should have, but modules are still coded to 2.7 compatibility, so that’s what I’ll use.)
Getting pip on is a bit more complicated – you have to install easy_install, then use that to install pip. So, I used the lovely directions at StackOverflow for inspiration:
- Grab setuptools from pypi.python.org/pypi/setuptools#files – make sure the version you get matches the version of Python installed – in my case, 2.7
- Install setuptools. You can follow the defaults, just make sure you install to the correct directory – where you installed Python. This should be auto-detected though.
- Open Powershell (I’m running Win 7; if you aren’t, you should be. And if you can’t run W7, open up command prompt instead.)
- In the shell, change to the directory where you installed Python, and then to the Scripts directory in that folder – ie.
./easy_install pip– this works because setuptools added an executable called easy_install.exe to the Scripts/ folder
- Pip is now installed. If you want to install something with Pip (i.e. ), open up Powershell again if it’s not already open, change to that directory, and run
./pip install requests
For bonus points, and to get Python to run without having to prepend the directory where Python is installed to every command you run with Python, append the Python directory to your environment PATH variable.
How do you do this? On Win 7, type “Path” into the search bar in the start menu. It should get you something like this:
Select the “Edit system environment variables” option. Not the one with “Your account”.
That will take you to this screen. See the button close to the bottom labeled “Environment Variables”? That’s the one you want to click. And when you do that, you’ll get this:
I’ve skipped a bit, but you want to scroll though the box on the bottom to find the Path variable. When you’ve found it, either double click on it, and single click to select it, and press Edit.
When the screen comes up, hit “End” on your keyboard to jump to the end of the line, then add a semi-colon (which is the ; symbol, if you don’t know), and paste the directory where you installed Python. Or type it. Copy & Pasting directly from an Explorer window is less error prone, so that’s what I do.
Because it’s a system variable, it’ll only take global effect when you restart your computer. If you just want to use Python in Powershell though, just open a new Powershell instance. You can verify that Python’s present in your Path by typing
$env:Path and looking at the end of the line that gets printed.
Amazon markets Spot Instances as a way to reduce the price you pay for instances. So, continuing my efforts to reduce expenses on EC2, I looked into using spot instances. Spot instances are essentially just like normal instances. You can create your own AMIs, where you essentially create an image and tell Amazon to create instances based on that image, or use an existing AMI.
If you want to create an AMI, get a starting image, and customize it as necessary. I started with the Fedora 17 image. In an attempt to reduce the cost, I resized the disk from 10GB to 2GB, installed vim, less, screen and rsync, which oddly aren’t in the default Fedora install.
I then had to package it as a new AMI – this created an EBS snapshot, so I’m happy that I resized the disk. It’s a bit annoying that you’re going to be paying for an EBS snapshot AND the active EBS volumes, but in virtually all cases, the cost of the EBS snapshot won’t exceed the amount saved by using spot instances. If you have a bigger snapshot, it’ll cost more of course, but then you’d likely be using a more expensive EC2 instance, so the cost should balance out in the long run.
As for actually using the spot instance, I had my AMI set up to automatically start an IRC bot, so I used this for timing. The IRC bot came online ~7 minutes after I submitted the request to start the spot instance, so there’s a bit of lead time, but not too much.
Because of the lead time, the instance won’t appear in the instance list for a while.
And an extra tip: Don’t be like me and not realise the spot instance actually started, and leave it running for two months racking up charges, only to be notified by Amazon that you now owe them money after your credit runs out. (Thankfully, they waived the charges as a one time thing.)
So now by default I set an expiry time of a day on all my spot instance requests if I know I’m only going to have them up for a few hours.
And one thing to look at if you require access to your data and can get by with using a pre-created image is using instance stores and mounting EBS volumes with the API. I didn’t try it because apparently, the t1.micro size that I’m using doesn’t support instance stores. Of course, this only really makes sense if you don’t want to pay the cost of having the spot instance run off an EBS volume. For a large scale operation, could be worth it.