Radek Antoniuk warden IT Tech Entrepreneur personal notepad

Building and testing multi sub-project .NET solutions

Assuming that we have a bigger Microsoft .NET project that has a structure like this:

ROOT
  -- SubProject1
  -- SubProject1\SubProject1.sln
  -- SubProject1.Tests
  -- SubProject1.Tests\SubProject1.Tests.csproj
  -- SubProject2
  -- SubProject2\SubProject1.sln
  -- SubProject2.Tests
  -- SubProject2.Tests\SubProject2.Tests.csproj
  -- SubProject3
  -- SubProject3\SubProject1.sln
  -- SubProject3.Tests
  -- SubProject3.Tests\SubProject2.Tests.csproj

we can build it at once using MSBuild. Lets create a builder.proj file

<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" >
<ItemGroup>
	<Solution Include="**\*.sln" />
</ItemGroup>
<ItemGroup>
	<TestAssemblies Include="**\**\bin\**\*Tests.dll"/>
</ItemGroup>
<Target Name="Build">
  <MSBuild Projects="@(Solution)" Targets="Build"/>
</Target>
<Target Name="Test">
  <Exec Condition=" '@(TestAssemblies)' != ''"
          Command="Mstest.exe /resultsfile:results.trx @(TestAssemblies ->'/testcontainer:"%(RecursiveDir)%(Filename)%(Extension)"', ' ')"
          ContinueOnError="true" IgnoreExitCode="true" />
</Target>
</Project>

and then we can execute

msbuild /t:Build builder.proj -- building the project
msuilbd /t:Test builder.proj -- running all tests

References

Copying Artifacts from Bamboo Deployment

Sometime we face the situation that we need to copy the deployment results (or Artifacts) to some central repository where all deployments are stored. This is not a problem for Linux-to-Linux copy as we can use SCP, RSYNC or any other task. What if the target is a Windows machine though and if we want to customize the directory naming to be dependent on e.g. version of the product hidden in the exe file? PowerShell to the rescue! We need two pieces:

We need a Bamboo Remote Agent that is running on a Windows box - this will be doing the deployment itself and executing the PowerShell script.

A powershell script Task (normal Script task with the Powershell type selected) and a script that will do the actual copy

$TODAY=(Get-Date).ToString('yyyy-MM-dd')
$VERSION=(dir Setup.Full.exe).VersionInfo.ProductVersion
$DIR=&quot;Z:\Releases\ProductName\$VERSION-$TODAY-${bamboo.deploy.release}\&quot;
Write-Host &quot;Saving artifacts to $DIR&quot;
New-Item -itemtype directory $DIR
Copy-Item * $DIR

What this script does is:

It sets current date to $TODAY variable It takes setup.full.exe and pulls out version information from inside It create a new directory in $DIR path Copies all files from the current deployment bamboo agent directory to the target $DIR

No Debian Wheezy 7.0 XenServer template? Don't worry!

There is no template for Debian Wheezy while creating new VMs in XenServer. However, it is easy to clone it from the 6.0 version. Firstly we clone the template, then we change the parameters to use Wheezy.

# xe vm-clone uuid=`xe template-list name-label="Debian Squeeze 6.0 (64-bit)" --minimal` new-name-label="Debian Wheezy 7.0 (64-bit)"
# xe template-param-set other-config:default_template=true other-config:debian-release=wheezy uuid=`xe template-list name-label="Debian Wheezy 7.0 (64-bit)" --minimal`

Consuming WSDL Web Services with Perl or Python

If you ever try to talk from a script with Web Service that via WSDL definition, you may end up with a script like this:

#!/usr/bin/perl
use SOAP::Lite +trace => 'debug';
#use SOAP::Lite;
use Data::Dumper;
use strict;
use warnings;
my $serviceWsdl = 'http://server/service?wsdl';
my $soap = SOAP::Lite
  ->service($serviceWsdl);
my $result = $soap->serviceMethod();
unless ($result->fault) {
	print $result->result();
} else {
	print join ', ',
	      $result->faultcode,
	      $result->faultstring;
}

This will work with some web services, but with others it will not, especially when the method invoked on the service is parameter-less. The resulted XML code will show xsi:nil=”true” in the method invocation, what will effectively break the query. To overcome this, I just switched to Python and came up with this simple script:

from suds.client import Client
client = Client('http://server/service?wsdl')
print client.service.serviceMethod()

Puppet: send an email to the client when a new key is generated

Puppet is great for centralised management of SSH keys on Linux boxes. The SSH module described in the project pages does its job really well for creating a new key pair and distributing it for clients (using the keys) and servers (authorized_keys file management).

The key generation mechanism provides several options to set up how the keys should be generated. One of them, is the maxdays options, defining how long the keys are valid. Because of that, I needed some mechanism to notify the users when their key has changed and they need to fetch new one. Normally, ssh::auth::server can be used for private key distribution, sometimes however this is not possible and that’s why this need floated.

To solve that, I thought that the simplest way will be to send an email that is set in the account properties. Basically the most crucial code needed is this one:

   exec { "Create key $title: $keytype, $length bits":
      command => "ssh-keygen -t ${keytype} -b ${length} -f ${keyfile} -C \"${keytype} ${length}\" -N \"\"",
      user    => "puppet",
      group   => "puppet",
      creates => $keyfile,
      require => File[$keydir],
      before  => [File[$keyfile, "${keyfile}.pub"],Exec["Notify user ${email}"]]
    }
    exec { "Notify user ${email}":
        command => "cat ${keyfile} | mail -s 'New SSH key notification' ${email}",
        subscribe => Exec["Create key $title: $keytype, $length bits"],
        refreshonly => true
    }

The command for user notification is in the exec clause and it is using a simple mail command.

The full diff of modules/ssh/manifests/auth.pp is below:

Index: modules/ssh/manifests/auth.pp
===================================================================
--- modules/ssh/manifests/auth.pp
+++ modules/ssh/manifests/auth.pp
@@ -22,7 +22,7 @@
 # is done in the private definitions called by the virtual resources:
 # ssh_auth_key_{master,server,client}.                                
-define key ($ensure = "present", $filename = "", $force = false, $group = "puppet", $home = "", $keytype = "rsa", $length = 2048, $maxdays = "", $mindate = "", $options = "", $user = "") {
+define key ($ensure = "present", $filename = "", $force = false, $group = "puppet", $home = "", $keytype = "rsa", $length = 2048, $maxdays = "", $mindate = "", $options = "", $user = "", $email="") {                                                                                                                                                    
   ssh_auth_key_namecheck { "${title}-title": parm => "title", value => $title }
@@ -44,6 +44,7 @@
     length  => $_length,
     maxdays => $maxdays,
     mindate => $mindate,
+    email   => $email
   }
   @ssh_auth_key_client { $title:
     ensure   => $ensure,
@@ -144,7 +145,7 @@
 # This definition is private, i.e. it is not intended to be called directly by users.
 # ssh::auth::key calls it to create virtual keys, which are realized in ssh::auth::keymaster.
-define ssh_auth_key_master ($ensure, $force, $keytype, $length, $maxdays, $mindate) {
+define ssh_auth_key_master ($ensure, $force, $keytype, $length, $maxdays, $mindate, $email) {
   Exec { path => "/usr/bin:/usr/sbin:/bin:/sbin" }
   File {
@@ -212,8 +213,14 @@
       group   => "puppet",
       creates => $keyfile,
       require => File[$keydir],
-      before  => File[$keyfile, "${keyfile}.pub"],
-    }
+      before  => [File[$keyfile, "${keyfile}.pub"],Exec["Notify user ${email}"]]
+    }
+
+    exec { "Notify user ${email}":
+       command => "echo 'A new SSH key for you has just been generated. Please fetch it from the server.' | mail -s 'New SSH key notification' ${email}",
+       subscribe => Exec["Create key $title: $keytype, $length bits"],
+       refreshonly => true
+    }
   } # if $ensure  == "present"

Ctrl+# commit shortcut and other ones don't work in Eclipse

{"options"=>{}}

Yii:: Avoid to include jquery twice in AJAX rendered actions

An idea to solve that is to use ClientScript and disable the scripts that you want to avoid. To do that automatically and globally, you can use a custom controller like this:

# components/MyController.php 
<?php
abstract class MyController extends CController {
	public function beforeAction(){
		if( Yii::app()-&gt;request-&gt;isAjaxRequest ) {
			Yii::app()-&gt;clientScript-&gt;scriptMap['jquery.js'] = false;
			Yii::app()-&gt;clientScript-&gt;scriptMap['jquery-ui.min.js'] = false;
		}
		return true;
	}
}

Debian I/O disk problem when rebooting on XenServer guest

Problem:

  • You are using a XenServer machine that is running kernel somewhere around version 2.6.32
  • You have a guest system like Debian Wheezy (currently testing) that is running kernel 3.0+
  • While rebooting your guest system (VM) you are experiencing disk issues like I/O access, ext4 inconsistency, etc…
  • The disks on guest are not mounting properly or are mounted in read-only mode or fsck has problems checking them

Solution:

In the HOST logs you can see probably:

Apr 10 11:04:14 server kernel: blkfront: xvda: barriers enabled

The problem is that 3.0 kernels are not using barriers anymore and there is a bug connected to that with the host kernel. If you are interested in details, check out Debian bug #637234.

The workaround is to use ‘barriers=0’ while mounting the disks in the guest system, so you correct your /etc/fstab to have:

/dev/xvda1        /        ext4      errors=remount-ro,barrier=0      0     1

Monitoring Tomcat PermGen, OldGen and Threads with Nagios nrpe

Today I am sharing with you a Nagios Perl script for monitoring Tomcat through JMX. This script can be used for monitoring Tomcat server parameters through Nagios NRPE server. Script can be found here. It is not all parametrised with the command line, but can be easily adapted to monitor other parameters and/or display other PerfData. As you can see in the source, the script is checking for 3 different params for PermGen: - Perm Gen - PS Perm Gen - CMS Perm Gen This is because Tomcat can use different Garbage Collectors (see this answer).

Running Redmine under unicorn in Debian

This post explains how to set up Redmine - the packaged version in Debian, to be hosted by Unicorn server and to use apache load balancing module. This is a faster alternative to running Redmine with mod_passenger.

The idea is as follows: unicorn is starting X servers running Redmine, and Apache is load-balancing the requests between those servers. We have to start with installation of unicorn gem

gem install --no-ri --no-rdoc unicorn

Now, we have to creation Unicorn configuration file:

working_directory "/usr/share/redmine"
pid "/tmp/redmine.pid"
preload_app true
timeout 60
worker_processes 4
listen 4000
GC.respond_to?(:copy_on_write_friendly=) and GC.copy_on_write_friendly = true
after_fork do |server, worker|
        #start the worker on port 4000, 4001, 4002 etc...
        addr = "0.0.0.0:#{4000 + worker.nr}"
        # infinite tries to start the worker
	server.listen(addr, :tries => -1, :delay => -1, :backlog => 128)
        #Drop privileges if running as root
        worker.user('www-data', 'www-data') if Process.euid == 0
endand so on...

Before we try to start Redmine, we have to create a symlink to the logs directory:

ln -s /var/log/redmine/default/ /usr/share/redmine/log

Now, we can start Redmine and test if it works:

/usr/bin/unicorn_rails -E production -c /etc/redmine.ru --path '/redmine'

Note: Change the /redmine path to the path you want your Redmine should be visible at. This is needed for correct URL operations. If everything is running smoothly, you can start the server with su and add the line to /etc/rc.local:

su -c "/usr/bin/unicorn_rails -E production -c /etc/redmine.ru --path '/support' -D" www-data

This is starting Unicorn initially as www-data user. Apache 2 Balancer configuration In this step, we tell Apache how to route requests to Redmine between the workers. Firstly, let’s enable the appropriate modules:

a2enmod balancer proxy proxy_http

Next, edit an appropriate VirtualHost configuration file. In the default configuration it would be /etc/apache2/sites-available/default and add the corresponding lines:

   SetEnv X_DEBIAN_SITEID "default"
    <Proxy balancer://redmine>
        BalancerMember http://127.0.0.1:4000
        BalancerMember http://127.0.0.1:4001
        BalancerMember http://127.0.0.1:4002
        BalancerMember http://127.0.0.1:4003
    </Proxy>
    Alias "/redmine/plugin_assets/" /var/cache/redmine/default/plugin_assets/
    RewriteEngine On
    RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_URI} !^.*plugin_assets.*$
    RewriteRule ^(/redmine/.*)$ balancer://redmine$1 [P,QSA,L]
    <Location /redmine/sys>
        Order deny,allow
        Deny from all
        allow from 1.2.3.4
        allow from 1.2.3.5
    </Location>

A little bit of description:

The SetEnv part is needed for Redmine Debianized configuration to choose the instance of Redmine Proxy section defines the balancing workers to be used, be sure to adjust this if you are starting less/more worker threads The Alias line is needed for some static files used by Redmine The Rewrite section eventually tells Apache to route all Redmine requests to the load balance by matching /redmine/ against $REQUEST_URI variable. The Location part is for the Redmine Web Service security. Adjust this to limit the access to the /redmine/sys/ only for the trusted servers.

Deleting duplicate records in MySQL

Sometimes you have multiple records where the same data spans across multiple columns in the table and you’d like to clean up that mess by leaving only one copy per unique set of records.

Let’s consider the following recordset:

mysql> select * from testdup;
+----+------+------+------+
| id | a    | b    | c    |
+----+------+------+------+
|  1 | AAA  | BBB  | CCC  |
|  2 | AAA  | BBB  | CCC  |
|  3 | AAA  | BBB  | CCC  |
|  4 | AAA  | BBB  | CCC  |
|  5 | AAA  | BBB  | CCC  |
|  6 | AAA  | BBB  | CCC  |
|  7 | AAA  | BBB  | CCC  |
|  8 | AAA  | BBB  | CCC  |
|  9 | AAA  | BBB  | CCC  |
| 10 | AAA  | BBB  | CCC  |
| 11 | AAA  | BBB  | CCC  |
| 12 | CCC  | DDD  | AAA  |
| 13 | CCC  | DDD  | AAA  |
| 14 | CCC  | DDD  | AAA  |
| 15 | CCC  | DDD  | AAA  |
| 16 | CCC  | DDD  | AAA  |
| 17 | CCC  | DDD  | AAA  |
| 18 | CCC  | DDD  | AAA  |
| 19 | CCC  | DDD  | AAA  |
| 20 | CCC  | DDD  | AAA  |
| 21 | CCC  | DDD  | AAA  |
| 22 | CCC  | DDD  | AAA  |
| 23 | AAA  | BBB  | CCC  |
+----+------+------+------+
23 rows in set (0.00 sec)

Now we want to clean it up to have the result as following:

 
|  1 | AAA  | BBB  | CCC  |
| 12 | CCC  | DDD  | AAA  |

This is quite simple. Firstly we will create a table containing only the records we want to create the pristine

mysql> select * from testdup_clean;
+----+------+------+------+
| id | a    | b    | c    |
+----+------+------+------+
|  1 | AAA  | BBB  | CCC  |
| 12 | CCC  | DDD  | AAA  |
+----+------+------+------+
2 rows in set (0.00 sec)

This is our table that contains the clean record set. Now, if you have unique ID field in the original table you can remove the unwanted records with the following query:

DELETE FROM testdup WHERE id NOT IN ( SELECT id FROM testdup_clean);

Otherwise, if you don’t have the ID field, you can just truncate the original table and insert the records from the clean table with:

 
mysql> TRUNCATE testdup;
Query OK, 0 rows affected (0.02 sec)
mysql> INSERT INTO testdup SELECT * from testdup_clean;
Query OK, 2 rows affected (0.00 sec)
Records: 2  Duplicates: 0  Warnings: 0
mysql> select * from testdup_clean;
+----+------+------+------+
| id | a    | b    | c    |
+----+------+------+------+
|  1 | AAA  | BBB  | CCC  |
| 12 | CCC  | DDD  | AAA  |
+----+------+------+------+
2 rows in set (0.00 sec)

Hope that helps somebody :-)

Quickies Installing Global Rvm Passenger 3 Ree Rails3 In A Debian


layout: post title: RVM, Passenger 3, REE, Rails3 in a Debian date: 2011-01-03 03:16:23.000000000 +00:00 categories: - Linux - Rails tags: - passenger - ree - ruby - rvm – This post explains how to install a global RVM (Ruby Version Manager) with Passenger 3 gem, Ruby Enterprise Edition in a Debian server machine for all users.

  1. Create /etc/rvmrc with the contents:
export rvm_path=&quot;/opt/rvm&quot;
export rvm_source_path=&quot;${rvm_path}/src&quot;
export rvm_log_path=&quot;${rvm_path}/log&quot;
export rvm_bin_path=&quot;${rvm_path}/bin&quot;
export rvm_gems_path=&quot;$rvm_path/gems&quot;
export rvm_tmp_path=&quot;${rvm_tmp_path:-&quot;$rvm_path/tmp&quot;}&quot;
export rvm_install_on_use_flag=0
export rvm_gemset_create_on_use_flag=0
# export rvm_make_flags=&quot;-j7&quot;
export rvm_trust_rvmrcs_flag=1
export rvm_pretty_print_flag=1

I have chosen /opt/rvm/ for the installation. 2. To install RVM, use the cool script provided on rvm’s site:

bash &lt; &lt;(curl http://rvm.beginrescueend.com/releases/rvm-install-head)
  1. To enable RVM in the command-line, add this to /etc/profile at the very end:
[[ -s &quot;/opt/rvm/scripts/rvm&quot; ]] &amp;&amp; source &quot;/opt/rvm/scripts/rvm&quot;

Be sure to modify the path /opt/rvm if you have installed rvm elsewhere. 4. Re-login to enable rvm command. Now we have RVM environment installed and we will take care of Ruby Enterprise Edition. 1. Installing Ruby Enterprise Edition

rvm install ree
  1. Installing Passenger 3.
rvm ree
#installing the gem
gem install passenger
#compiling Apache2 module
passenger-install-apache2-module
  1. Now add/change the paths how Passenger is loaded in /etc/apache2/mods-available/ in the files passenger.load and passenger.conf:
#passenger.load
LoadModule passenger_module /opt/rvm/gems/ree-1.8.7-2010.02/gems/passenger-3.0.2/ext/apache2/mod_passenger.so
#passenger.conf
PassengerRoot /opt/rvm/gems/ree-1.8.7-2010.02/gems/passenger-3.0.2
PassengerRuby /opt/rvm/wrappers/ree-1.8.7-2010.02/ruby

Next enable the module with a2enmod passenger and restart apache. You can verify if Passenger started with grep:

server:~# ps aux | grep Passenger
root     10515  0.0  0.3   4164  1676 ?        Ssl  02:25   0:00 PassengerWatchdog
root     10519  0.0  0.3  12868  1920 ?        Sl   02:25   0:00 PassengerHelperAgent
root     10521  0.1  1.5  17720  7812 ?        S    02:25   0:00 Passenger spawn server
nobody   10524  0.0  0.6   9504  3088 ?        Sl   02:25   0:00 PassengerLoggingAgent
  1. Next, we will make the REE the default environment and create a default gemset.
#switch to REE
rvm use ree
#create the gemset
rvm gemset create default
#use the gemset and set it as default
rvm use ree@default --default
#install rails3 in the gemset
gem install rails3

If you want normal users to be able to create gemsets and install additional gems, you have to change permissions of /opt/rvm/gems/. This is not very elegant and super-safe solution, but for development machines is ok.

chmod -R 777 /opt/rvm/gems/

Using RVM in CRON If you try to use RVM in cron the normal way, like:

37 * * * *      /bin/bash -l -c &quot;RAILS_ENV=production /opt/rvm/bin/rake db:backup;&quot;

You will end up with one of the following errors:

command rvm is not found command rake is not found gem XXX is not found

This is due to the fact, that cron commands are run in a very limited environment whereas RVM heavily relies on the environment variables. As a remedy for that situation, you have to invoke bash as if it was invoked by a login shell:

38 * * * * 	/bin/bash -l -c &quot;cd project; RAILS_ENV=production rake db:backup;&quot;

Explanation:

-l is invoking bash with full environment for RVM to work properly. Remember to put the appropriate RVM loading line in the .bashrc ! -c says read the command from the parameter next in the command, you cd to the project directory; this is needed to set up proper RVM environment from the .rvmrc file - so yes, I am assuming you have a file named .rvmrc in your project directory inside which you are selecting the proper gemset

Mysql gem under MacOS Snow Leopard

To set up this software under your Applebook, you have to download MySQL first. Go to MySQL download page and fetch an appropriate version for your system, depending if it 32 or 64 bits.
I'm having MacBook Pro with Intel Core 2 Duo.
Ergo, this is 64-bit architecture, ergo, I've chosen to download Mac OS X ver. 10.6 (x86, 64-bit), DMG Archive.

Can’t create global fcgid-proctbl mutex

You probably see this error when you are trying to start apache with /etc/init.d/apache2 start and there is no effect.
This error is caused by not clearing semaphores by apache and/or mod_fcgid and/or mod_passenger.
You can see the list of semaphores with the command ipcs -s.

Rails application crashing when trying to upload files with mod_passenger

{"options"=>{}}

Mysql::Error: query: not connected with Rails ActiveRecord

{"options"=>{}}

hints: using screen in linux

I am working a lot with screen, especially to get to remote hosts and do many operations at once. The thing that was irritating me a lot, was the fact that I had to set the titles of windows manually after logging in via ssh.

I have found a method to overcome this problem though. Firstly, you have to put this line in your ~/.bashrc file.

PROMPT_COMMAND='echo -ne &quot;&#92;&#48;33k$HOSTNAME&#92;&#48;33\\&quot;'

Secondly, to get a nice status bottomline when you start screen, you have to set it in ~/.screenrc. I am using this set of customizations:

# UTF-8 everywhere
defutf8 on
# disable Ctrl-S/Ctrl-Q &quot;flow control&quot;
defflow off
# skip intro
startup_message off
# detach on disconnect
autodetach on
# use regular audible bell
vbell off
# use backtick for prefix key instead of Ctrl-A
#escape ``
# print wackier status messages
nethack on
# restore window contents after using (e.g.) vi
altscreen on
#fancy statusbar
#shelltitle &quot;$ |bash&quot;
hardstatus alwayslastline '%{= kg}[ %{G}%H %{g}][%= %{= kw}%?%-Lw%?%{=b kR}(%{W}%n*%f %t%?(%u)%?%{=b kR})%{= kw}%?%+Lw%?%?%= %{g}][%{Y}%l%{g}]%{=b C}[ %d %M %c ]%{W}'

Nothing better than a screenshot huh? Ok, here you are, THE RESULT:

Simple authentication in Zend Framework

This is quick example of how to do a very simple authentication in Zend Framework. It doesn't user external database for getting the credentials, although you can make it very easily to read data in the Auth adapter from an external file or MySQL database.

Synchronizing Mailman lists with LDAP

Quite common usage scenario is a Postfix mail server with a Mailman instance for groups. Even more often the accounts for the mail are stored in an LDAP tree. And in this case a problem appears, because Mailman doesn’t have a native connection to LDAP. That’s why I needed to have an LDAP 2 Mailman script that would synchronise mailman groups with the LDAP tree. This Perl script is meant to be run from cron.

What this script does is the following:

  • it binds to the LDAP server with the credentials provided at the beginning of the script
  • it searches for objects of class mailGroup in the organizationalUnit ou=lists
  • gets commonName (cn) of the object
  • checks if the list of that name exists in mailman already
  • gets all mail attributes values and all memberdn’s mail attributes values and updates the mailing list’s members with them

If the list does not exist, and $create_nonexistent is true, then a new list will be automatically created with the admin/password defined in the variables at the beginning of the script. Note: For security reasons, this version of the script DOES NOT delete lists, so if you delete a mailGroup from LDAP, you have to do a rmlist -a by yourself. On request, I can give somebody the version which deletes the lists also.

The script relies on a mailGroup objectClass which is defined in the schema that you can download from here.

#!/usr/bin/perl
###### LDAP-2-Mailman Synchronization script ######################################
# @author: Radek Antoniuk
# @website: http://www.warden.pl
# @license: GNU
#
# This script synchronises (adds/removes) the users from the Mailman mailing lists
# according to the objects found in LDAP. You can put it in cron. The script was
# written using Debian-style paths to Mailman, you may need to adjust them
# if you have a different mailman installation.
#
###################################################################################
use strict;
use warnings;
use Net::LDAP;

my $ldap_host = "ldap://localhost";
my $ldap_bind_dn = "cn=ldapuser,dc=domain,dc=com";
my $ldap_bind_pass = "password";
my $ldap_base_dn = "ou=lists,dc=domain,dc=com";

my $create_nonexistent = 1;
my $default_list_admin = 'admin@domain.com';
my $default_list_password = "111111";

# Connect to LDAP proxy and authenticate
my $ldap = Net::LDAP->new($ldap_host) || die "Can't connect to server\n";
$ldap->bind($ldap_bind_dn, password => $ldap_bind_pass) || die "Connected to server, but couldn't bind\n";

# search for interesting groups
my $ret = $ldap->search( base   => $ldap_base_dn,  filter => "(&(objectClass=mailGroup))" );
die "Search returned no groups\n" unless $ret;

print "\n\n";
print "------------------------------\n";

foreach my $group ($ret->entries) {

  my $member_emails = ""; #list of emails in the group
  my $list_name = $group->get_value("cn");

  if($list_name) {
    print "Processing list: $list_name \n";

    # get the membership list
    my @member_list = $group->get_value("member");

    # make a list of emails to pass to mailman from member objects
    foreach my $member_dn (@member_list) {
            my $person = $ldap->search(  base  => $member_dn, filter => "(&(mail=*)(!(pwdAccountLockedTime=*)))", scope => "base"  );
            # try to get the referred object, or continue if locked account or object does not exist #
            if (my $member = $person->entry(0)) {
               my $email = $member->get_value("mail");
               $member_emails .= $email."\n";
            }
    };

    # now process normal mail attributes #
    @member_list = $group->get_value("mail");

    foreach my $email (@member_list) {
            $member_emails .= $email . "\n";
    };

    # check if list exists
    if (! -d "/var/lib/mailman/lists/$list_name" ){
        print "List $list_name does not exist.\n";

        #if not and we want to create it automatically
        if ($create_nonexistent) {
                print "Creating new list $list_name.\n";
                qx{/usr/lib/mailman/bin/newlist -q $list_name $default_list_admin $default_list_password};
                # check if now the list exists
                die "FATAL: Unable to create list $list_name" if (! -d "/var/lib/mailman/lists/$list_name" );
        }

    }

    print "\nSyncing $list_name...\n";

    open( PIPE, "|/usr/lib/mailman/bin/sync_members -w=yes -g=yes -a=yes -f - $list_name" ) || die "Couldn't fork process! $!\n";
    print PIPE $member_emails;
    close PIPE;

    print "------------------------------\n";
  };
};

$ldap->unbind;
print "\n\n";

Postfix SASL and LDAP Authorization

Hints for Postfix 2.6 or higher that are commonly mentioned problems or scenarios.

Enforcing From: header to match SASL login username There are many situations where we have to make sure that the user that is trying to send e-mail is the rightful person. In practice, this means that a user that is logging in via username via SASL mechanism has to own the e-mail address that she is trying to send as. To accomplish that, firstly we want to put an additional header to the mails being sent. This is done by putting into main.cf:

smtpd_sasl_authenticated_header=yes

This results in an additional header in mails, that are sent by SASL authenticated users:

(Authenticated sender: sasl_username)

Note: this is NOT required and this is potentially exposing your existing usernames to the world, making them easier to bruteforce or to spam. In this scenario however, this is anyway meaningless because from address == SASL username. Next, we want to make sure that if the user is trying to send mail as name.surname@domain.com, then he has to be authenticated as name.surname via SASL. For that, we have to put:

smtpd_sender_restrictions = reject_sender_login_mismatch
smtpd_sender_login_maps = ldap:/etc/postfix/login-maps.cf&lt;

In my case, I am authorising the SASL users against LDAP directory, thus the login-maps.cf contains:

version = 3
server_host = localhost
search_base = dc=domain,dc=com
bind = yes
bind_dn = cn=user,dc=domain,dc=com
bind_pw = password
query_filter = (&amp;amp;(uid=%u)(objectClass=inetOrgPerson))
result_attribute = uid

What this map does, is it looks up in LDAP directory the SASL username (uid=%u) looking in objects of class InetOrgPerson. For success, the lookup must return:

username           name.surname@domain.com

where name.surname@domain.com is the email that the user is trying to send as (from From: header).

PHP LDAP change password page

{"options"=>{}}

Creating Debian Lenny Bootable USB key in Windows

{"options"=>{}}

datePicker and jqModal z-index problem

{"options"=>{}}

QuickTip: Styling jqModal with blockUI and AJAX requests

{"options"=>{}}

Grepping Oracle's database dumps = awk hints.

{"options"=>{}}

How to fully update Windows XP from command-line?

{"options"=>{}}

Networked Unattended WinXP installation

Hola!

Recently I have been doing an XP preinstallation for a remote software deployment project. In this post i'm putting all the interesting links I have found and the problems I had during the software preparation.

This is NOT a howto or instruction, this is just intended to help. If you want to have a full howto, don't hesitate to comment/fund a coffee for me :)

Mantis tweaks: logging in via Microsoft Active Directory LDAP

This is a short article on how to get the above Mantis version to work with Microsoft Windows Active Directory LDAP. Used Mantis version: 1.1.1 and PHP Version 5

Mantis currently supports only login via LDAP directory by the following scheme:

Connect to LDAP server Bind with anonymous DNS or with a user specified DN (but in a config file permanently) If the bind succeeds then do an ldap_search If the search succeeds then login is successful.

When using Microsoft AD LDAP the situation is a bit different, we want to:

Connect to LDAP server

Create a DN basing on the config file and username field that the user entered in the login form Try Bind with the above DN and password that the user entered in the login form If the bind succeeds then the login is successful (we don’t need to run the search)

To achieve that, there are some minor changes to do in the Mantis core API.

Step 1

Log in to a fresh mantis installation, and create a user with admin privileges with a username matching your LDAP username (in this example xy2093)

Step 2

First, add to your config_inc.php configuration file the following options:

/* we want to use LDAP auth */

$g_login_method = LDAP;
$g_ldap_server = 'ldap://ldap.myhost.com/';

/* the root DN that will be used to form the bind DN during authentication phase */
$g_ldap_root_dn = 'ou=staff,ou=company,dc=domain,dc=com';

/* we don't want the users to be able to sign-up via mantis */
$g_allow_signup=OFF;

/* we want to use Mantis email field instead of LDAP one */
$g_use_ldap_email = OFF;

/* we don't want false mantis lost password feature */
$g_lost_password_feature = OFF;

Step 3:

Next, you have to modify the core LDAP authentication ldap_authenticate function. Go to core/ldap_api.php, find the above function and replace it with:

function ldap_authenticate( $p_user_id, $p_password ){
if (is_blank($p_password))
return false;
$t_ldap_host = config_get('ldap_server');
$t_ldap_port = config_get('ldap_port');
$t_ldap_rdn = config_get('ldap_root_dn');
$t_ds = ldap_connect($t_ldap_host, $t_ldap_port) or die('Unable to connect to LDAP server&lt;br /&gt;');
$t_user = user_get_field($p_user_id, 'realname'); //This checks the users Real Name instead of username
$t_uname = user_get_field($p_user_id, 'username');
$binddn = "CN=$t_user ($t_uname),$t_ldap_rdn";
$t_authenticated = false;
if(@ldap_bind($t_ds,$binddn,$p_password))
$t_authenticated = true;

return $t_authenticated;
}

In the function notice the $bind_dn variable. This is the variable being used to prepare the bind DN for LDAP connection. Feel free to modify it to suit your authentication scheme, however you should not have to. It defaults to:

CN=Firstname Lastname (username),ou=staff,ou=company,dc=domain,dc=com

i.e. CN=John Doe (xy2093),ou=staff,ou=company,dc=domain,dc=com.

You won’t believe it but that’s it! Now you can try to log in to Mantis with your LDAP password and it should work like a charm.

The next issue to solve here is that you have to have the users from LDAP in your $mantis_user_table, for instance to manage Mantis privileges. There are many ways to achieve that, you can import them every night. Or you can use Mantis SOAP API to check if the user exists in LDAP when they try to log in as I did. But how to do that is another article ;)

Is this a four level domain or an alias?

{"options"=>{}}

FTP behind NAT with TLS howto

Ever wondered how to set up a more secure FTP server?
I did. And the first thing that came to my mind was getting those username/password things encrypted.
Who did invent a plaintext login mechanism!?
Sheeesh...

So, let's get started.
This howto will tell you how to set up an Proftpd based FTP server with TLS encryption for data&control channels.

XEN PAE with LVM on IA-64

{"options"=>{}}