January 29, 2016

How to fix InsecurePlatformWarning on Ubuntu?

Python modules sometimes give issues. We got below warning from a python application.

/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:120: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.

After a small research we found out, this might be an issue related to outdated module. We ran below commands in that server. After that warning didn't occur.

$ sudo apt-get install build-essential python-dev libffi-dev libssl-dev
$ sudo pip install --upgrade ndg-httpsclient

January 14, 2016

Vagrant on Windows 7 vs Ubuntu 14.04

My whole team had to work on a project which is using Vagrant. Most of us had 8GB memory except one unfortunate intern. He had only 4GB of memory on his workstation. All the team members could spawn Vagrant machines without a problem except him.

So we requested for more memory. Insisted IT department to upgrade it to 8GB. Oh no! Our IT is going to retire desktops. So they don't buy new parts for existing desktop system. Somehow we managed to get 1GB memory card. Now he got 5GB memory in his computer. This computer had Athlon processor. ( I cannot recall the model number.)

Then we tried to spin it up again. To provision to vagrant machine it took at least 3 hours. Sometimes package gets corrupted. Somehow he stopped to provision machines, once he realized that it is useless.

Then we moved him to another task. There he had to work with Ubuntu closely. So I forced him to install Ubuntu. However this kid was okay to install Ubuntu. Then I created Ubuntu 14.04 USB bootable drive. Then he installed Ubuntu. After installation he just tried to start the Vagrant. Vagrant machine got provisioned within minutes. Still system has 1GB+ free memory.

Windows 7 vs Ubuntu 14.04, Ubuntu 14.04 wins when it compares with memory consumption.  

January 05, 2016

How to specify ReleaseLabel for EMR cluster with Boto2

Boto is the AWS SDK for Python. You can create clusters, instances or anything using Boto. But sometimes Boto imposes limitations. I wanted to create a EMR cluster with RelaseLabel 4.2.0. But we were using Boto2. ReleaseLabel is an option in Boto3. For Boto2 there was no documented option for RelaseLabel.

So I found out a way to create EMR (Elastic Map Reduce) clusters using Boto 2 with a given ReleaseLabel.

I have commented AMI Version because ReleaseLabel will pick AMI version correctly. Above program will print the cluster ID in terminal. 

Sometimes you might get an issue saying "No Default VPC found.". This is a network related issue. In that case you might need to specify subnet ID for EMR cluster. Then you don't need to specify an availability zone.