Installing Ansibe from source

Installing Ansible from source (with fish shell)

1. Check python version python --version if python is not installed you should install it.

2. Install easy_install and pip sudo apt-get install python-setuptools sudo easy_install pip

3. Install c compiler sudo apt-get install build-essential and python dev sudo apt-get install python-dev

4. And install dependencies sudo pip install paramiko PyYAML Jinja2 httplib2 six

5. Move to directory where you want to install ansible (this should be directory that user has acces to because he will need to run set env script) ~/lib and execute git clone git://github.com/ansible/ansible.git --recursive

6. Go into directory cd ./ansible

7. Set running config script on login into shell (~/config/fish/config.fish)

   # Run script to set ansible to work
   # > redirects stdout and ^ redirects stderr
   . /home/mladen/lib/ansible/hacking/env-setup.fish -q > /dev/null ^ /dev/null
   

8. You should also add script to get latest version of ansible from git to /etc/init.d/

   I named script refresh-ansible

   #!/bin/sh
   
   # Help for setup: https://wiki.debian.org/LSBInitScripts
   
   ### BEGIN INIT INFO
   # Provides:          refresh-ansible
   # Required-Start:    $remote_fs $syslog
   # Required-Stop:     $remote_fs $syslog
   # Default-Start:     3
   # Default-Stop:      
   # Short-Description: Start refresh-ansible at boot time
   # Description:       Refreshes ansible instalation at boot time.
   ### END INIT INFO
   
   cd /home/mladen/lib/ansible
   git pull --rebase
   git submodule update --init --recursive
   

9. You should make this file executable sudo chmod +x refresh-ansible and run command sudo update-rc.d refresh-ansible defaults. This will add script to directory /etc/rc3.d/. If you later want to remove script from boot just run sudo update-rc.d refresh-ansible remove

10. Next you need to install boto sudo -H pip install -U boto

11. Create ~/.boto file with content

    [Credentials]
    aws_access_key_id = AKIAJTKAWVR5TELVR4VA
    aws_secret_access_key = wir9eklBfm8HQbDIlkxfIkOOpRb3VJ7mGFgVgu4a
    

12. Change .boto permissions chmod 400 ~/.boto

13. Last you need to put your aws credentials in .ssh directory (*.pem)

Maven Archetypes ++

When I'm trying out code samples I find it useful to create Maven archetypes. I find it useful to create maven modules that have all properties set in parent project. This saves me time when trying out code from chapters that have a lot of examples whit similar configurations.

This link explains a lot, but I have few additions.

1. If you'r using Eclipse (STS) to create archetype you should take care not to use embedded maven.
2. After you create archetype with mvn archetype:create-from-project you should edit pom.xml in target/generated-sources/archetype
1. Change version so it won't be SNAPSHOT
2. Adjust name, artifactId and description.
3. You should also brows sub-directories to see if there are hidden files there (.settings/, .projec or .classpath). You should remove this files.
4. You should also check pom.xml in target/generated-sources/archetype/src/main/resources/archetype-resources/ to see if there are any problems there.

Only when your satisfied with state of your project should you run mvn install from target/generated-sources/archetype/.

If you' using Eclipse tu create new project with newly created archetype, make sure that you select Default Local as catalog, and if you didn't remove SNAPSHOT from version you should check Include snapshot archetypes checkbox.

Logback blocks when running on Java 7 (on Linux)

I have noticed that my JUnit tests take noticeable time. After debuging test I have found that logback library has this problem. As you can see in comments this is not bug with logback but with Java 7. But every time I run JUnit logback block for 10 second.

Solution is to edit /etc/hosts and add IP address and hostname as described here

Addendum to Atlassian migration guide SVN->Mercurial

If you have unconventional SVN layout like this:

 FOO
 |
 |--branches
 |--tags
 |--trunk
 |--foodir
     |-src
     |-...

Atlassian recommendation is to use svn-dump-reloc but that didn't help me in this case.

You could manually copy foodir to trunk but that will lose all svn history on files when migrated to Mercurial.

What I did is this:

1. Created local SVN repository

 mkdir svnRepo  
 cd svnRepo/  
 svnadmin create svnload  
 svnadmin load svnload/ < ~/Documents/Mercurial/svn.dump  

2. Run Mercurial convert command but with explicitly set directory for trunk

 hg --config convert.svn.trunk=foodir convert file:///tmp/Mercurial/svnRepo/svnload/FOO "hgRepos/FOO" --authormap authors.txt  

Also Migration guide doesn't say but you should alter .hgrc file and put:

 [extensions]  
 convert =  

Monitoring Tomcat with VisualVM

VisualVM is great tool that you can use to monitor and profile java applications. In this post I will explain how to use VisualVM to monitor Tomcat. You should never reconfigure Tomcat memory setup without first checking where is the problem.

Installing VisualVM

Download zip from website.  Unzip content into desired directory and run visualvm.exe

Monitoring Tomcat in Eclipse

Open launch configuration of Server that you wish to monitor and to VM Arguments add:

 -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=9090 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false   

Now if you start Tomcat can connect to it with VisualVM.

Monitoring Tomcat as a Windows Service

You should open Configuration Editor with:

 tomcat7w.exe //MS//  

and add:

 -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=9090 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false   

to Java Options

Monitoring Tomcat in standalone mode

Open catalina.bat or catalina.sh (depending on the environment) and add:

 -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=9090 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false   

to JAVA_OPTS.

Final Comment on VisualVM

If you want to monitor common java programs from eclipse you can install VisualVM launcher in eclipse. Details are described here.

Useful links

When problems with memory arise you can debug them using next two links:
http://www.skill-guru.com/blog/2010/09/23/how-to-increase-heap-size-in-tomcat-6/
http://www.skill-guru.com/blog/2010/10/05/increasing-permgen-size-in-your-server/

Setting Tomcat for development and production

In this post I will describe how to setup great Tomcat installation.

Installation

Download Tomcat for your environment from official download site. Don't download Windows Service Installer even if you plan to use Tomcat as service.

Unzip Tomcat to desired directory.

Changing logging engine to Logback

After installing Tomcat I will change logging to use Logback following this instructions.

• copy everything from tomcat-slf4j-logback/bin to destination/bin (this will overwrite tomcat-juli.jar)
• copy logging configuration in tomcat-slf4j-logback/conf to destination/conf
• delete logging.properties from destination/conf
• copy logback-core.jar and logback-access.jar to destination/lib

Change Valve in server.xml with:

 <Valve className="ch.qos.logback.access.tomcat.LogbackValve" quiet="true" filename="${catalina.home}/conf/logback-access-localhost.xml" />  

After that you should run tomcat with startup.bat to check that everything works.

Setting Tomcat users

In directory conf/ you should edit tomcat-users.xml. Add this inside tomcat-users tags:

 <role rolename="manager-gui"/>  
 <role rolename="admin-gui"/>  
 <user username="tomcat-mng" password="*****" roles="manager-gui,admin-gui"/>  

Adding Tomcat to Eclipse

Add new Tomcat server as usually. First add Server runtime enviroment (Windows->Preferences), and then add new Server using that runtime.
You should adjust Publishing and Timeouts as preferred. I use "Never publish automatically" , start 450s and stop 150s. 
After that you should set "Launch Configuration", in VM arguments you should add:

 -Djuli-logback.configurationFile="<absolute tomcat home path>\conf\logback.xml"  

In Server project on your workspace you should check that server.xml has right Valve set.

Now you can start server to check that everything is OK.

Running Tomcat as Windows Service

You should install Tomcat as a service using service.bat. Advanced setup is described here.

 service.bat install  

You should run this command as an admin. This will add Tomcat to your services, but will not enable monitor service and edit configuration service. So I use this command:

 tomcat7w.exe //MS//   

You install service with specific name:

 service.bat install [service_name]  

Then you need to rename tomcat7.exe and tomcat7w.exe to [service_name].exe and [service_name]w.exe respectively. You should run this programs as administrator.

Now you can change setup so Tomcat will be started automatically. Also you should add

 -Djuli-logback.configurationFile="<absolute tomcat home path>\conf\logback.xml"  

to Java Options.

Redirecting Stdout and Strerr

When you run Tomcat in production you should redirect stdout and stderr to null.

Connecting Tomcat with PostgreSQL on OpenShift

Now that we have created Tomcat on OpenShift we can use one of the Gears to install Database server. One of the advantages of OpenShift is that it allows user to create full flagged RDBS for free. I will explain here how to install PostgreSQL but you can choose MySQL or MongoDB if you need NoSQL database.

Creating a gear

You should log in your account on OpenShift and select My Applications tab and click on a name of your application. This will take you to configuration page.


Now you should choose Add Cartridge and select PostgreSQL database. On the next screen you should just click Add Cartridge. After initialization is done you will be prompted connection parameters for your new database and instructions how can you remove this cartridge. You should take note of information on this screen because once you close it you will not be able to get that information.

If you followed my instructions PostgreSQL will be installed on a same Gear as Tomcat so you will still have 2 free Gears.

Connecting Tomcat and PostgreSQL

If you want to connect to your new database from Eclipse or pgAdmin you need to start port forwarding and that can be only done from RHC tool and is only enabled while you run port forwarding command (rhc port-forward -a tomcat).
If you install MySQL database you can use  phpMyAdmin interface for interaction with database, but I prefer PostgreSQL.

Next we need to copy PostgreSQL JDBC driver to Tomcat /lib directory, I used WinSCP for this, or you can do it trough git if you followed my instructions from last post.

You need to add resource setting to global section of server.xml

 <Resource   name="jdbc/tomcat"   
       auth="Container" type="javax.sql.DataSource"   
       driverClassName="org.postgresql.Driver"   
       url="jdbc:postgresql://OPENSHIFT_INTERNAL_IP:5432/tomcat"   
       username="admin"   
       password="****"   
       maxActive="20"   
       maxIdle="10"   
       maxWait="-1"/>  

And also add in context.xml:

 <ResourceLink name="jdbc/tomcat" global="jdbc/tomcat"  
     type="javax.sql.DataSource" />  

After that you should restart server.

You should note that I put OPENSHIFT_INTERNAL_IP in DB url and this constant will be changed in start script that is in action_hooks directory. If you are not using GIT for managing Tomcat configuration you should set this to real IP address.

You can check internal IP with command: echo $OPENSHIFT_INTERNAL_IP
(You shoud be logged in with putty), or you can note what IP is displayed when you port forward.

Getting data source in Java is out of the scope of this post, but you can check out my demo program on GitHub at https://github.com/McNullty/db-tester.

There are examples of connecting to database with Apache BasicDataSource and using JNDI.