Monday 27 October 2014

Git log search for term or follow file

I've been tracking down file changes quite a bit lately in git.. Here's how:

Look for a specific word or term

git log -g --grep=SEARCH_THISgit log -SSEARCH_FOR+THIS

Look for changes to a specific file?

git log --follow somefile/somewhe.re
gitk somefile/somewhe.re
git log -p somefile/somewhe.re

git log -pfollow -p file

Isn't tracking down whodunnits fun?

Wednesday 27 August 2014

python 2.7 on centOS 5.8

These are my dropbox notes on steps taken to install python 2.7 on a centOS 5.8 box.

Python 2.4 is used by default for yum on centOS 5.x. Replace that with a new version and yum will no longer work. Do not upgrade the default python. You'll see that I place python 2.7 in /opt/bin/
yum groupinstall "Development Tools"
wget http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tgz
tar xvfz Python-2.7.3.tgz
cd Python-2.7.3

#sqllite module does not build: I hope this gist is still around..
curl -sk https://raw.github.com/gist/2727063/ | patch -p1

./configure --enable-ipv6 --prefix=/opt/bin/python2.7 --with-threads --enable-shared --enable-unicode
make
make install

su -c 'ln -s /opt/bin/python2.7/lib/libpython2.7.so /usr/lib'
su -c 'ln -s /opt/bin/python2.7/lib/libpython2.7.so.1.0 /usr/lib'
su -c '/sbin/ldconfig -v'

cd /opt/bin/python2.7/lib
export PATH=`pwd`:$PATH
cd /opt/bin/python2.7/bin
export PATH=`pwd`:$PATH

#needed --no-check-certificate for this next wget..
wget http://pypi.python.org/packages/source/s/setuptools/setuptools-0.6c11.tar.gz#md5=7df2a529a074f613b509fb44feefe74e --no-check-certificate
tar xf setuptools-0.6c11.tar.gz
cd setuptools-0.6c11

more README.txt 
#read it
more EasyInstall.txt
#read the multiple python versions bit.

python2.7 setup.py install
easy_install-2.7 pip

pip-2.7 install virtualenv
pip-2.7 install anything else you need

echo done.


done.

Wednesday 13 August 2014

Using ansible for environment alignment and configuration

ansible is an IT automation tool for server environment alignment, configuration and ops type work.

The use case: fetch a zip from the client's server and run the contents of that zip over multiple nix boxes in the development environment after production releases have occurred.

To configure logging, make sure to set the path to the ansible log file:

ANSIBLE_LOG_PATH=/path/to/ansible.log
export ANSIBLE_LOG_PATH

Once ansible has been installed, either using the source or your distro's package managed, go to the terminal and setup a host file, named hosts

localhost ansible_python_interpreter='/usr/bin/python'

[myboxes]
10.0.0.1 ansible_ssh_user=usernamehere ansible_ssh_pass=usernamepasshere ansible_sudo_pass=ThisdoesntWorkNotInSudoers ansible_su_pass=SuPasswordhere

[databaseservers]
# boxa
10.0.0.2 ansible_ssh_user=user ansible_ssh_pass=what ansible_sudo_pass=what ansible_su_pass=what
# boxb
10.0.0.3 ansible_ssh_user=user ansible_ssh_pass=what ansible_sudo_pass=what ansible_su_pass=what

Now do not try run ansible commands if simplejson for python is not on the target machine. simplejson is a python package and may already be installed depending on your python version and environment.

If simplejson is not installed you can use the ansible raw module to execute a command without depending on simplejson being installed

>ansible -i hosts myboxes -m raw -a "yum -y install python-simplejson" --sudo -u yourname

That is only going to work if I (`whoami`) am in sudo. In this instance I am not.

I need su.
ansible -i hosts myboxes -m raw -a "yum -y install python-simplejson" --su
This works because I configured the host file with a password for each box using the configuration key ansible_su_pass. I should use ssh key pairs and I will.. but the beauty of it is I don’t have to. I can just provide the passwords in the file. Must save that file somewhere safe..

Here's the output:

Dependencies Resolved

================================================================
 Package                  Arch          Version               Repository   Size
================================================================
Installing:
 python-simplejson        x86_64        2.0.9-8.el5           base        141 k

Transaction Summary
================================================================
Install       1 Package(s)
Upgrade       0 Package(s)

Total download size: 141 k
Downloading Packages:
python-simplejson-2.0.9-8.el5.x86_64.rpm                 | 141 kB     00:00
...Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing     : python-simplejson                                        1/1

Installed:
  python-simplejson.x86_64 0:2.0.9-8.el5

Complete!

Didn't quite get it? Using the command ansible -i hosts myboxes -m raw -a "yum -y install python-simplejson" --su resulted in the successful installation of python-simplejson.
Now that simplejson is installed, here's a couple of sample ansible commands:

ansible -i hosts databaseservers -m raw -a "locate *.repo" --su

ansible -i hosts databaseservers -a "mkdir ~/ops/deps"

ansible -i hosts databaseservers -m copy -a "src=./test.txt dest=~/ops/deps"

Of course these are simple examples, and more complex sets of steps can be executed as playbooks – recipes for sys admin type tasks. I'll write a follow up on playbooks.

Being able to run a command over multiple machines at once is definitely better than opening 5 shells.

 

Wednesday 23 July 2014

Jetty 9.1 maven plugin error Unsupported major.minor version 51.0 when compiling for Java 6

Running a jetty server via the maven plugin gave me this error recently.

jetty run mojo Unsupported major.minor version 51.0

The stack trace looked like this:

Caused by: java.lang.UnsupportedClassVersionError: org/eclipse/jetty/maven/plugin/JettyRunMojo : Unsupported major.minor version 51.0
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
    at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClassFromSelf(ClassRealm.java:386)
    at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:42)
    at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:244)
    at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:230)
    at org.sonatype.guice.bean.reflect.URLClassSpace.loadClass(URLClassSpace.java:101)
    ... 41 more


It turns out that if you are compiling for java 6 and you use jetty maven plugin v9.x you will get this unsupported version error. This is because the jetty maven plugin for v9.0 or higher requires java 7.

So change your jetty maven plugin in your pom to use Jetty version 8.1 if you need to target java 6.

Tuesday 22 July 2014

centOS 5 error Address family not supported by protocol when performing yum update

You're running 'yum update' on a centOS 5 box and get this error:

Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=5&arch=x86_64&repo=os error was
[Errno 4] IOError: <urlopen error (97, 'Address family not supported by protocol')>
Error: Cannot find a valid baseurl for repo: base
The solution is that you need to setup your proxy, so open up a terminal and type as follows:

http_proxy="http://username:password@proxyurl:port/
export http_proxy
yum update
You should find that the yum command is executed without error:
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: centos.joinweb.co.il
 * extras: centos.joinweb.co.il
 * updates: centos.joinweb.co.il
base                                                     | 1.1 kB     00:00
base/primary                                             | 1.3 MB     00:24
base                                                                  3662/3662

curl usage or how to post to a webservice from the commandline


curl is super for talking to http from the command line.

HTTP GET

In general, to use curl to do a GET request do as follows:

curl -v -H "header:value" http://host:port/webservice/uri

HTTP POST

To POST to a RESTful webservice, in this instance a JAX-RS Jersey REST service running on Jetty on an IBM AIX box:

  • Using curl 7.30.x from Windows
curl -H "header:value" -X POST -d "{json}" http://host:port/webservice/postResource
Note that you may have to escape the double quote in a json string, e.g. '{\"Key\":,\"Value\"}'  when posting the JSON data right on the command line.
  • Using curl 7.9.3 on the IBM AIX, the POST argument aint gonna fly:
curl -H "header:value" -X POST -d "{json}" http://host:port/webservice/postResource

Here's a tip - Passing the json on the command line is actually not great, its tough to read and tough to write out.

Rather create a file for the json packet and pass as an argument to curl like this:
curl -H "header:value" -X POST -d @myfilename.txt http://host:port/webservice/postResource

Wednesday 16 July 2014

Install curl on an IBM AIX

Get curl onto an IBM AIX. Here's how.

It is pretty simple - download curl from the IBM site
then run:
su -c rpm -Uvh curl-7.9.3-2.aix4.3.ppc.rpm
 done.