Article
Suchen
Shameless Plugs
Most Popular
Who's online
Online users
User login
5329
Build your own Scripts to Manipulate LDIF Exports - Part 3
Putting Everything Together
Table of Contents
Introduction
Content of the attached archive
The LDIFStruct library
Python
Vim
Php
Perl
BeanShell
Java
Mono/C#
How to use the library
Loading and merging LDIF files
Cycling through the structure
Re-exporting to LDIF format
Exporting to XML format
Exporting to CSV format
Exporting to SQL formats
Summary
Introduction
In the first part of the article, Build your own scripts to manipulate LDIF exports - Part 1, you discovered how to use associative arrays and lists, and in the second part of the article, Build your own scripts to manipulate LDIF exports - Part 2, you discovered how to read from file, parse command-line arguments and use regular expressions in different languages.
In this part, you will see some details about the following features:
- Creating a class (for Perl, Php, BeanShell, Java and Mono/C#)
- Loading LDIF exports using associative arrays and lists
- Looping through the different structures to generate content (XML, LDIF, CSV, SQL, etc.)
You will see details about the LDIFStruct library that manipulates LDIF exports in the following languages:
- Python ( http://www.python.org/about/)
- Perl ( http://www.perl.org/about.html)
- Php ( http://www.php.net/)
- Vim ( http://www.vim.org/about.php)
- BeanShell ( http://www.beanshell.org/intro.html)
- Java ( http://java.sun.com/)
- Mono/C# ( http://www.mono-project.com/Main_Page)
Content of the attached archive
Here is the content of the file LDIFStruct_part3.zip:
./LDIFStruct
|__ docs
| |__ article_part3.txt
| \__ article_part3.html
|__ Python
| |__ ldifstruct.py
| |__ test_library.py*
| |__ file1.ldif
| \__ file2.ldif
|__ Php
| |__ LDIFStruct.php
| |__ test_library.php*
| |__ file1.ldif
| \__ file2.ldif
|__ Vim
| |__ LDIFStruct.vim
| |__ test_library.vim*
| |__ test_library.sh*
| |__ file1.ldif
| \__ file2.ldif
|__ Java
| |__ LDIFStruct.jar
| |__ LDIFStruct.java
| |__ LDIFStruct.class
| |__ test_library.sh*
| |__ file1.ldif
| \__ file2.ldif
|__ Mono
| |__ LDIFStruct.cs
| |__ LDIFStruct.exe*
| |__ test_library.sh*
| |__ file1.ldif
| \__ file2.ldif
|__ Perl
| |__ LDIFStruct.pl
| |__ test_library.pl*
| |__ file1.ldif
| \__ file2.ldif
\__ BeanShell
|__ LDIFStruct.bsh
|__ test_library.bsh*
|__ test_library.sh*
|__ file1.ldif
|__ file2.ldif
\__ bsh-2.0b4.jar
Details:
test_library.*: executable scripts to test the LDIFStruct library for the different languagesLDIFStruct.*, ldifstruct.*: the LDIFStruct library for the different languages. This library has been used in the Automatic Role Matrix and in the Automatic Tree Reports articles*.ldif: sample LDIF files to test the scripts. Most of the scripts will take these two files as arguments, so you can for instance use the following command to test the Python library:./test_library.py file1.ldif file2.ldifarticle_part3.txt: the Wiki source of this articlearticle_part3.html: the result of the conversion from Wiki to HTML (see Wiki to CoolSolutions Converter)
The LDIFStruct library
To build a structure in memory form a LDIF file, we need to analyze each line of the file, handle new object DNs and store them in the structure, create attribute lists and add values to them. The pseudo-code to load an LDIF export in a data structure would be the following:
for each line of the LDIF export | if line has a LDIF format XXX: YYY | | if XXX is "dn" | | | add the dn entry in the associative array | | else if line contains an attribute value | | | if attribute exists for current dn | | | | if value does not exist yet in attributes | | | | | add value in attribute values | | | | end if | | | else | | | | create a new attribute entry for the current dn entry | | | | add value in attribute values | | | end if | | end if | end if end for
Now let's have a look at how it is implemented in the different languages.
1. The first part of the library deals with imports and the LDIFStruct class declaration:
import os import re import sys class LDIFStruct:
2. The second part reads a LDIF file, joins LDIF multi-lines and stores everything in a list (it works the same as in the second part of the article):
def load( self, dic, ldiffile ):
# Reading file in a list and join LDIF multi-lines
file = open( ldiffile )
data = []
try:
for line in file:
match = re.match( '^ (.*\n)', line )
# Check if line begins with a space, join with last line
if match:
# Pop last line and remove line return
lastline = re.sub( '(.*)\n', '\\1', data.pop() )
# Join result with current line without space
newline = lastline + match.group(1)
data.append( newline )
# Normal line
else:
data.append( line )
finally:
file.close()
3. After the data is loaded in memory, the last part analyzes each line and creates the final dictionary (one entry per DN, for each DN one dictionary per attribute and for each attribute one list of values):
# Loop through lines and analyse content
currentdn = ""
for line in data:
# Test if we have a typical LDIF line "xxx: yyy"
match = re.match( '^(.*?): (.*)\n', line )
if match:
# Check if line contains a DN
if match.group(1) == "dn":
currentdn = match.group(2)
if not dic.has_key( currentdn ):
dic[ currentdn ] = {}
# Check if line contains an attribute
elif not re.match( 'version|changetype|add|replace|delete', match.group(1) ):
if not dic[ currentdn ].has_key( match.group(1) ):
dic[ currentdn ][ match.group(1) ] = [ match.group(2) ]
else:
if not match.group(2) in dic[ currentdn ][ match.group(1) ]:
dic[ currentdn ][ match.group(1) ] += [ match.group(2) ]
1. The first part reads a LDIF file, joins LDIF multi-lines and stores everything in a list (it works the same as in the second part of the article):
" Load a LDIF export in a dictionary from current buffer.
"
" Params:
" dic - Dictionary variable
function! LDIF:Load(dic)
try
let s:data = []
let s:index = 0
while s:index < line('$')
let s:index = s:index + 1
let s:line = getline(s:index)
if matchstr(s:line,'^ .*') != ''
"Join LDIF multi-lines
let s:currentline = substitute(s:line,'^ \(.*\)','\1','')
let s:prevline = s:data[-1]
let s:data[-1] = s:prevline.s:currentline
else
call add(s:data, s:line)
endif
endwhile
2. After the data is loaded in memory, the last part analyzes each line and creates the final dictionary (one entry per DN, for each DN one dictionary per attribute and for each attribute one list of values):
"Analyse LDIF content, attributes and dn are stored in a dictionary
let s:currentdn = ''
let s:totaldn = 0
let s:totalattr = 0
let s:index = 0
while s:index < len(s:data)
let s:line = s:data[s:index]
let s:index = s:index + 1
" Test if we have a typical LDIF line 'xxx: yyy'
let s:elements = matchlist(s:line,'^\(.\{-}\): \(.*\)')
if s:elements != []
"Check if current line contains a new dn
if s:elements[ 1 ] == 'dn'
let s:currentdn = s:elements[ 2 ]
if !has_key( a:dic, s:currentdn )
let a:dic[s:currentdn] = {}
endif
"Check if current line contains a new attribute
elseif matchstr(s:elements[ 1 ],'^version\|^changetype\|^modify\|^add\|^replace\|^delete') == ''
"Check if attribute already exist in dic entry
if has_key( a:dic[s:currentdn], s:elements[ 1 ] )
let s:allattrvalues = a:dic[s:currentdn][s:elements[ 1 ]]
"Add value in values list if not existing
if count( s:allattrvalues, s:elements[ 2 ] ) == 0
call add( s:allattrvalues, s:elements[ 2 ] )
endif
else
"Create a new dic entry for the attribute
let a:dic[s:currentdn][s:elements[ 1 ]] = [s:elements[ 2 ]]
endif
endif
endif
endwhile
catch
echo 'Error while executing script'
endtry
endfunction
1. The first part of the library deals with the LDIFStruct class declaration:
<?
class LDIFStruct
{
2. The second part reads a LDIF file, joins LDIF multi-lines and stores everything in a list (it works the same as in the second part of the article):
static function load( $array, $LDIFFile )
{
// Check if file exists
if (!file_exists($LDIFFile))
{
printf("ERROR: File '$LDIFFile' does not exist!\n");
exit(1);
}
// Read file content in an array
$data = array();
$fd = fopen ($LDIFFile, "r");
while (!feof ($fd))
{
$buffer = fgets($fd, 4096);
if ( preg_match( '/^ .*/', $buffer ) )
{
// Join LDIF multi-lines
$currentline = preg_replace( '/^ /', '', $buffer );
$prevline = array_pop( $data );
$prevline = preg_replace( '/\n/', '', $prevline );
array_push( $data, $prevline.$currentline );
}
else
{
array_push( $data, $buffer );
}
}
fclose ($fd);
3. After the data is loaded in memory, the last part analyse each line and create the final dictionary (one entry per DN, for each DN one dictionary per attribute and for each attribute one list of values):
// Analyse LDIF content, attributes and dn are stored in an associative array
$currentdn = "";
foreach ($data as $line)
{
if ( preg_match( '/^(.*?): (.*)\n/', $line ) > 0 )
{
$elem1 = preg_replace( '/^(.*?): (.*)\n/', '\1', $line );
$elem2 = preg_replace( '/^(.*?): (.*)\n/', '\2', $line );
// Check if line contains a DN
if ( $elem1 == "dn" )
{
$currentdn = $elem2;
if ( !array_key_exists( $currentdn, $array ) )
{
$array[ $currentdn ] = array();
}
}
// Check if line contains an attribute
elseif ( preg_match( '/^version|changetype|^modify|^add|^replace|^delete/', $elem1 ) == 0 )
{
// Check if attribute already exist in array entry
if ( array_key_exists( $elem1, $array[ $currentdn ] ) )
{
// Add value in values array if not existing
$allattrvalues = $array[ $currentdn ][ $elem1 ];
if ( !in_array( $elem2, $allattrvalues ) )
{
$array[ $currentdn ][ $elem1 ][] = $elem2;
}
}
else
{
// Create a new array entry for the attribute
$array[ $currentdn ][ $elem1 ] = array( $elem2 );
}
}
}
}
return $array;
}
}
?>
Perl
1. The first part of the library concerns the LDIFStruct class declaration:
package LDIFStruct;
2. The second part reads a LDIF file, joins LDIF multi-lines and stores everything in a list (it works the same as in the second part of the article):
sub load
{
my ( $array, $LDIFFile ) = @_;
# Open LDIF file
open( DAT, $LDIFFile ) || die( "Could not open file ".$LDIFFile."!" );
@raw_data = <DAT>;
close( DAT );
my @data = ();
foreach my $line ( @raw_data )
{
if ( $line =~ /^ (.*)/ )
{
# Join LDIF multi-lines
$currentline = $1;
$prevline = pop( @data );
$prevline =~ s/\n//;
push( @data, $prevline.$currentline."\n" );
}
else
{
push( @data, $line );
}
}
3. After the data is loaded in memory, the last part analyzes each line and creates the final dictionary (one entry per DN, for each DN one dictionary per attribute and for each attribute one list of values):
# Analyse LDIF content, attributes and dn are stored in an associative array
my $currentdn = "";
foreach my $line ( @data )
{
if ( $line =~ /^(.*?): (.*)\n/ )
{
# Check if line contains a DN
if ( $1 eq "dn" )
{
$currentdn = $2;
if ( !exists( $array{ $currentdn } ) )
{
$array{ $currentdn } = {};
}
}
# Check if line contains an attribute
elsif ( !( $1 =~ /^version|^changetype|^modify|^add|^replace|^delete/ ) )
{
# Check if attribute already exist in array entry
if ( exists( $array{ $currentdn }->{ $1 } ) )
{
@allattrvalues = @{ $array{ $currentdn }->{ $1 } };
$valueexists = 0;
foreach my $value ( @allattrvalues ) { if ( $value eq $2 ) { $valueexists = 1; } }
# Add value in values array if not existing
if ( $valueexists == 0 )
{
push @allattrvalues, $2;
$array{ $currentdn }->{ $1 } = [ @allattrvalues ];
}
}
else
{
# Create a new array entry for the attribute
$array{ $currentdn }{ $1 } = [ $2 ];
}
}
}
}
return %array;
}
1;
1. The first part of the library deals with the imports and the LDIFStruct class declaration:
import java.util.regex.*;
class LDIFStruct
{
2. The second part reads a LDIF file, joins LDIF multi-lines and stores everything in a list (it works the same as in the second part of the article):
static load( map, LDIFFile )
{
// Check if file exists
file = new File( LDIFFile );
if ( !file.exists() )
{
print( "ERROR: File '" + LDIFFile + "' does not exist!\n");
return map;
}
// Read file content in an array
line = "";
data = new ArrayList();
try
{
in = new BufferedReader(new FileReader(LDIFFile));
if (!in.ready())
throw new IOException();
while ((line = in.readLine()) != null)
{
if ( line.matches( "^ .*" ) )
{
// Join LDIF multi-lines
data.set( data.size() - 1, data.get( data.size() - 1 ) + line.substring( 1 ) );
}
else
{
data.add(line);
}
}
in.close();
}
catch (IOException e)
{
print("Error");
return map;
}
3. After the data is loaded in memory, the last part analyzes each line and create the final dictionary (one entry per DN, for each DN one dictionary per attribute and for each attribute one list of values):
// Analyse LDIF content, attributes and dn are stored in an associative array
currentdn = "";
for( line : data )
{
p = Pattern.compile( "^(.*?): (.*)" );
m = p.matcher( line );
if ( m.matches() )
{
elem1 = m.group(1);
elem2 = m.group(2);
// Check if line contains a DN
if ( elem1.equals( "dn" ) )
{
currentdn = elem2;
if ( !map.containsKey( currentdn ) )
{
map.put( currentdn, new HashMap() );
}
}
// Check if line contains an attribute
else if ( !elem1.matches( "^version|^changetype|^modify|^add|^replace|^delete" ) )
{
// Check if attribute already exist in array entry
if ( map.get( currentdn ).containsKey( elem1 ) )
{
// Add value in values array if not existing
allattravalues = map.get( currentdn ).get( elem1 );
if( !allattravalues.contains( elem2 ) )
{
map.get( currentdn ).get( elem1 ).add( elem2 );
}
}
else
{
// Create a new array entry for the attribute
allattrvalues = new ArrayList();
allattrvalues.add( elem2 );
map.get( currentdn ).put( elem1, allattrvalues );
}
}
}
}
return map;
}
}
1. The first part of the library deals with the imports and the LDIFStruct class declaration:
import java.io.*;
import java.util.*;
import java.util.regex.*;
public class LDIFStruct
{
2. The second part reads a LDIF file, joins LDIF multi-lines and stores everything in a list (it works the same as in the second part of the article):
public static Map<String,Map<String,List<String>>> load( Map<String,Map<String,List<String>>> map, String LDIFFile )
{
// Check if file exists
File file = new File( LDIFFile );
if ( !file.exists() )
{
System.out.println( "ERROR: File '" + LDIFFile + "' does not exist!\n");
System.exit(1);
}
// Read file content in an array
String dataline;
List<String> data = new ArrayList<String>();
try
{
BufferedReader in = new BufferedReader(new FileReader(LDIFFile));
if (!in.ready())
throw new IOException();
while ((dataline = in.readLine()) != null)
{
if ( dataline.matches( "^ .*" ) )
{
// Join LDIF multi-lines
data.set( data.size() - 1, data.get( data.size() - 1 ) + dataline.substring( 1 ) );
}
else
{
data.add(dataline);
}
}
in.close();
}
catch (IOException e)
{
System.out.println("Error");
System.exit(1);
}
3. After the data is loaded in memory, the last part analyzes each line and create the final dictionary (one entry per DN, for each DN one dictionary per attribute and for each attribute one list of values):
// Analyse LDIF content, attributes and dn are stored in an associative array
String currentdn = "";
for( String line : data )
{
Pattern p = Pattern.compile( "^(.*?): (.*)" );
Matcher m = p.matcher( line );
if ( m.matches() )
{
String elem1 = m.group(1);
String elem2 = m.group(2);
// Check if line contains a DN
if ( elem1.equals( "dn" ) )
{
currentdn = elem2;
if ( !map.containsKey( currentdn ) )
{
map.put( currentdn, new HashMap<String,List<String>>() );
}
}
// Check if line contains an attribute
else if ( !elem1.matches( "^version|^changetype|^modify|^add|^replace|^delete" ) )
{
// Check if attribute already exist in array entry
if ( map.get( currentdn ).containsKey( elem1 ) )
{
// Add value in values array if not existing
List<String> allattravalues = map.get( currentdn ).get( elem1 );
if( !allattravalues.contains( elem2 ) )
{
map.get( currentdn ).get( elem1 ).add( elem2 );
}
}
else
{
// Create a new array entry for the attribute
List<String> allattrvalues = new ArrayList<String>();
allattrvalues.add( elem2 );
map.get( currentdn ).put( elem1, allattrvalues );
}
}
}
}
return map;
}
}
1. The first part of the library deals with the imports and the LDIFStruct class declaration:
using System;
using System.IO;
using System.Text;
using System.Collections;
using System.Text.RegularExpressions;
public class LDIFStruct
{
2. The second part reads a LDIF file, joins LDIF multi-lines and stores everything in a list (it works the same as in the second part of the article):
public static Hashtable load( Hashtable table, string LDIFFile )
{
// Check if file exists
if ( !File.Exists( LDIFFile ) )
{
Console.WriteLine( "Could not find file " + LDIFFile + "!" );
Environment.Exit( 1 );
}
// Read file content in an array
string fileline;
ArrayList data = new ArrayList();
StreamReader reader = new StreamReader(new FileStream(LDIFFile, FileMode.Open));
while((fileline = reader.ReadLine()) != null)
{
if ( Regex.IsMatch( fileline, "^ " ) )
{
// Join LDIF multi-lines
data[ data.Count - 1 ] = data[ data.Count - 1 ] + fileline.Substring( 1 );
}
else
{
data.Add( fileline );
}
}
reader.Close();
3. After the data is loaded in memory, the last part analyzes each line and create the final dictionary (one entry per DN, for each DN one dictionary per attribute and for each attribute one list of values):
// Analyse LDIF content, attributes and dn are stored in an associative array
string currentdn = "";
foreach( string line in data )
{
if ( Regex.IsMatch( line, "^.*?: .*" ) )
{
string elem1 = Regex.Replace( line, "^(.*?): (.*)", "$1" );
string elem2 = Regex.Replace( line, "^(.*?): (.*)", "$2" );
// Check if current line contains a DN
if ( elem1.Equals( "dn" ) )
{
currentdn = elem2;
if ( !table.ContainsKey( currentdn ) )
{
table.Add( currentdn, new Hashtable() );
}
}
// Check if current line contains an attribute
else if ( !Regex.IsMatch( elem1, "^changetype|^modify|^add|^replace|^delete|^version" ) )
{
// Check if attribute already exist in array entry
if ( ((Hashtable) table[ currentdn ]).ContainsKey( elem1 ) )
{
// Add value in values array if not existing
ArrayList allattravalues = (ArrayList) ((Hashtable) table[ currentdn ])[ elem1 ];
if( !allattravalues.Contains( elem2 ) )
{
((ArrayList) ((Hashtable) table[ currentdn ])[ elem1 ]).Add( elem2 );
}
}
else
{
// Create a new array entry for the attribute
ArrayList allattrvalues = new ArrayList();
allattrvalues.Add( elem2 );
((Hashtable) table[ currentdn ]).Add( elem1, allattrvalues );
}
}
}
}
return table;
}
}
How to use the library
We build the LDIFStruct library using different languages, combining all the features explaines in the different parts of the article. But how do we use the results? This section shows, for the Python language, how to load, merge and convert LDIF files. You'll find examples for all the other languages in the source.
Loading and merging LDIF files
To load a LDIF file in a structure you first have to create this structure. You can have a look at the different test_library.* files to see how it is done. Here is the source code to create the structure and call the load method of the LDIFStruct library to load the data for Python:
...
...
mydic = {}
myLDIFStruct = LDIFStruct()
myLDIFStruct.load( mydic, sys.argv[ 1 ] )
...
...
Once a structure is loaded, you can re-use it to load and merge more data. So calling the load method multiple times on the same structure will merge the different LDIF files. Here is how it is done in Python:
...
...
mynewdic = {}
myLDIFStruct.load( mynewdic, sys.argv[ 1 ] )
myLDIFStruct.load( mynewdic, sys.argv[ 2 ] )
...
...
This feature can be very powerful if you have created different exports for different types of attributes and want to have only one LDIF export with all the data. You can also combine exports from different trees and convert them in another format, such as XML.
Once the data is loaded in a structure, you can cycle through it to convert it or directly access data in it (see first part of the article on how to access the structure).
The generic format of the final structure is the following:
myarray
{
"cn=jsmith,o=novell"
=> {
"givenName" => [ "John", "Henry" ],
"sn" => [ "Smith" ],
"cn" => [ "jsmith" ],
"objectClass" => [ "inetOrgPerson" ],
"mail" => [ "jsmith@mydomain1.com", "jsmith@mydomain2.com" ]
},
"cn=jdoe,o=novell"
=> {
"givenName" => [ "Jane", "Jennifer" ],
"sn" => [ "Doe" ],
"cn" => [ "jdoe" ],
"objectClass" => [ "inetOrgPerson" ],
"mail" => [ "jdoe@mydomain1.com", "jdoe@mydomain2.com" ]
}
}
You can then access values for a given attribute and a given DN:
myarray[ "cn=jdoe,o=novell" ][ mail ] == [ "jdoe@mydomain1.com", "jdoe@mydomain2.com" ]
One very interesting feature using the Python LDAP module is that you can create such structure directly by doing an LDAP search call (see Automatic Role Matrix and Automatic Tree Reports articles):
l = ldap.initialize( "ldap://" + options.ldap_server + ":" + options.ldap_port )
l.simple_bind_s( options.ldap_binddn, options.ldap_passwd )
scopes = { "sub": ldap.SCOPE_SUBTREE, "one": ldap.SCOPE_ONELEVEL, "base": ldap.SCOPE_BASE }
roles_users = dict( l.search_s( options.ldap_basedn, scopes[ options.ldap_scope ], options.ldap_filter, [ options.ldap_attribute ] ) )
The last line will create the exact same structure as the LDIFStruct library. You can then choose in your scripts if you want to use a LDIF export or directly query your directory.
The pseudo-code to cycle through the structure would be the following:
for each dn in the structure | do something with the dn | for each attribute of the dn | | do something with the attribute | | for each value of the attribute | | | do something with the value | | end for | end for end for
In Python, the code would be the following:
for dn in dic:
# Do something with dn
for attr in dic[ dn ]:
# Do something with attr
for value in dic[ dn ][ attr ]:
# Do something with value
Quite simple, isn't it?
Now we can have a look on how to use this code to convert the loaded structure into different formats such as XML, SQL, CSV or back to LDIF.
In the test_library.py script, you will see two functions that convert the structure into LDIF. The first one converts it to a LDIF creation export and the second to a LDIF update export.
Here is the source code for the two functions:
def toLDIFCreate( dic ):
# Convert to LDIF - Creation
print "# LDIF generated by Python"
print "version: 1\n"
for dn in dic:
print "dn: %s" % ( dn, )
print "changetype: add"
for attr in dic[ dn ]:
for value in dic[ dn ][ attr ]:
print "%s: %s" % ( attr, value )
print ""
def toLDIFUpdate( dic ):
# Convert to LDIF - Update
print "# LDIF generated by Python"
print "version: 1\n"
for dn in dic:
print "dn: %s" % ( dn, )
print "changetype: modify"
for attr in dic[ dn ]:
print "add: %s" % ( attr, )
for value in dic[ dn ][ attr ]:
print "%s: %s" % ( attr, value )
print "-"
print ""
Once data is loaded in the mynewdic variable (merge of two LDIF exports), you can call the functions using the following code:
toLDIFCreate( mynewdic )
toLDIFUpdate( mynewdic )
Here is the result of the first function when you call the script by passing into the parameters file1.ldif and file2.ldif (using ./test_library.py file1.ldif file2.ldif):
# LDIF generated by Python version: 1 dn: cn=group3,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data changetype: add owner: cn=member5,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data member: cn=member3,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data member: cn=member1,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data member: cn=member2,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data member: cn=member4,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data manager: cn=member7,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data cn: group3 objectClass: groupOfNames dn: cn=group1,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data changetype: add owner: cn=member4,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data member: cn=member2,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data member: cn=member3,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data member: cn=member5,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data member: cn=member4,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data manager: cn=member1,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data cn: group1 objectClass: groupOfNames dn: cn=group2,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data changetype: add objectClass: groupOfNames member: cn=member6,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data member: cn=member5,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data cn: group2
Here is the result of the second function:
# LDIF generated by Python version: 1 dn: cn=group3,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data changetype: modify add: owner owner: cn=member5,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data - add: member member: cn=member3,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data member: cn=member1,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data member: cn=member2,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data member: cn=member4,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data - add: manager manager: cn=member7,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data - add: cn cn: group3 - add: objectClass objectClass: groupOfNames - dn: cn=group1,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data changetype: modify add: owner owner: cn=member4,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data - add: member member: cn=member2,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data member: cn=member3,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data member: cn=member5,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data member: cn=member4,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data - add: manager manager: cn=member1,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data - add: cn cn: group1 - add: objectClass objectClass: groupOfNames - dn: cn=group2,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data changetype: modify add: objectClass objectClass: groupOfNames - add: member member: cn=member6,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data member: cn=member5,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data - add: cn cn: group2 -
In the test_library.py script, you will see a function that convert the structure into a sample XML file. You can easily change the code to generate the exact XML file you want to.
Here is the source code for this function:
def toXML( dic ):
# Convert to XML
print "<?xml version=\"1.0\"?>"
print "<entries>"
for dn in dic:
print "\t<entry dn=\"%s\">" % ( dn, )
for attr in dic[ dn ]:
print "\t\t<attribute name=\"%s\">" % ( attr, )
for value in dic[ dn ][ attr ]:
print "\t\t\t<value>%s</value>" % ( value, )
print "\t\t</attribute>"
print "\t</entry>"
print "</entries>"
print ""
Once data is loaded in the mydic variable, you can call the function using the following code:
toXML( mydic )
Here is the result of this function:
<?xml version="1.0"?>
<entries>
<entry dn="cn=group3,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data">
<attribute name="owner">
<value>cn=member5,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data</value>
</attribute>
<attribute name="member">
<value>cn=member3,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data</value>
<value>cn=member1,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data</value>
<value>cn=member2,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data</value>
</attribute>
<attribute name="manager">
<value>cn=member7,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data</value>
</attribute>
<attribute name="cn">
<value>group3</value>
</attribute>
<attribute name="objectClass">
<value>groupOfNames</value>
</attribute>
</entry>
<entry dn="cn=group1,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data">
<attribute name="owner">
<value>cn=member4,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data</value>
</attribute>
<attribute name="member">
<value>cn=member2,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data</value>
<value>cn=member3,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data</value>
</attribute>
<attribute name="manager">
<value>cn=member1,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data</value>
</attribute>
<attribute name="cn">
<value>group1</value>
</attribute>
<attribute name="objectClass">
<value>groupOfNames</value>
</attribute>
</entry>
<entry dn="cn=group2,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data">
<attribute name="objectClass">
<value>groupOfNames</value>
</attribute>
<attribute name="cn">
<value>group2</value>
</attribute>
</entry>
</entries>
In the test_library.py script, you will see a function that converts the structure into a CSV file. The first step before exporting the data is to get the list of all possible attributes:
def toCSV( dic ):
# Convert to CSV
allattrs = []
for dn in dic:
for attr in dic[ dn ]:
if not attr in allattrs:
allattrs.append( attr )
print '"dn";' + ";".join( ['"%s"' % f for f in allattrs] )
When all possible attributes are listed, we can cycle through the structure and try to get the values for each listed attribute:
for dn in dic:
values = []
for attr in allattrs:
value = ""
if dic[ dn ].has_key( attr ):
value = ",".join( dic[ dn ][ attr ] )
values.append( value )
print '"%s";' % ( dn, ) + ";".join( ['"%s"' % f for f in values] )
print ""
Once the data is loaded in the mynewdic variable, you can call the function using the following code:
toCSV( mynewdic )
Here is the result of the function:
"dn";"owner";"member";"manager";"cn";"objectClass" "cn=group3,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data";"cn=member5,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data";"cn=member3,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data,cn=member1,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data,cn=member2,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data,cn=member4,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data";"cn=member7,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data";"group3";"groupOfNames" "cn=group1,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data";"cn=member4,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data";"cn=member2,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data,cn=member3,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data,cn=member5,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data,cn=member4,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data";"cn=member1,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data";"group1";"groupOfNames" "cn=group2,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data";"";"cn=member6,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data,cn=member5,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data";"";"group2";"groupOfNames"
You can save this part into a .CSV file and open it/import it in your favorite spreadsheet tool, which is of course OpenOffice Calc ...
In the test_library.py script, you will see a function that converts the structure into SQL statements. For the .CSV export, the first step before exporting the data is to get the list of all possible attributes. Once that's done, we can cycle through the structure to generate the SQL statements. There is one function for generating INSERT statements and another for generating UPDATE statements:
def toSQLInsert( dic ):
# Convert to SQL - INSERT
allattrs = []
for dn in dic:
for attr in dic[ dn ]:
if not attr in allattrs:
allattrs.append( attr )
for dn in dic:
values = []
for attr in allattrs:
value = ""
if dic[ dn ].has_key( attr ):
value = ",".join( dic[ dn ][ attr ] )
values.append( value )
sqlfields = 'dn, ' + ", ".join( ['%s' % f for f in allattrs] )
sqlvalues = "'%s', " % ( dn, ) + ", ".join( ["'%s'" % f for f in values] )
print "INSERT INTO table ( %s ) VALUES ( %s );" % ( sqlfields, sqlvalues )
print ""
def toSQLUpdate( dic ):
# Convert to SQL - INSERT
allattrs = []
for dn in dic:
for attr in dic[ dn ]:
if not attr in allattrs:
allattrs.append( attr )
for dn in dic:
values = []
for attr in allattrs:
value = ""
if dic[ dn ].has_key( attr ):
value = ";".join( dic[ dn ][ attr ] )
values.append( value )
sqlfieldsvalues = dict( zip( allattrs, values ) )
updates = ", ".join( ["%s = '%s'" % ( k,v ) for ( k,v ) in sqlfieldsvalues.items()] )
print "UPDATE table SET %s WHERE dn = '%s';" % ( updates, dn )
print ""
Once the data is loaded into the mynewdic structure, you can export them to SQL using the following code:
toSQLInsert( mynewdic )
toSQLUpdate( mynewdic )
Here is the result returned by the first function:
INSERT INTO table ( dn, owner, member, manager, cn, objectClass ) VALUES ( 'cn=group3,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data', 'cn=member5,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data', 'cn=member3,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data,cn=member1,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data,cn=member2,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data,cn=member4,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data', 'cn=member7,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data', 'group3', 'groupOfNames' ); INSERT INTO table ( dn, owner, member, manager, cn, objectClass ) VALUES ( 'cn=group1,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data', 'cn=member4,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data', 'cn=member2,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data,cn=member3,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data,cn=member5,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data,cn=member4,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data', 'cn=member1,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data', 'group1', 'groupOfNames' ); INSERT INTO table ( dn, owner, member, manager, cn, objectClass ) VALUES ( 'cn=group2,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data', '', 'cn=member6,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data,cn=member5,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data', '', 'group2', 'groupOfNames' );
Here is the result returned by the second function:
UPDATE table SET owner = 'cn=member5,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data', member = 'cn=member3,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data;cn=member1,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data;cn=member2,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data;cn=member4,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data', manager = 'cn=member7,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data', cn = 'group3', objectClass = 'groupOfNames' WHERE dn = 'cn=group3,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data'; UPDATE table SET owner = 'cn=member4,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data', member = 'cn=member2,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data;cn=member3,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data;cn=member5,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data;cn=member4,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data', manager = 'cn=member1,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data', cn = 'group1', objectClass = 'groupOfNames' WHERE dn = 'cn=group1,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data'; UPDATE table SET owner = '', member = 'cn=member6,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data;cn=member5,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data', manager = '', cn = 'group2', objectClass = 'groupOfNames' WHERE dn = 'cn=group2,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data';
Summary
This is an overview on how to manipulate LDIF entries in different languages and how to use associative arrays and lists for that purpose. You can look at the different test_library.* files to see how to use the library in the different languages. You also discovered how to handle command line parameters, read lines from a file, build structures using associative arrays and lists and use regular expressions. (In the Reports article, you can also see how to use the powerful OptParser class to easily handle arguments and options on the command line.)
From there, you can re-use, improve and adapt the LDIFStruct library to your needs in your favorite language and use it to build new features. See also Automatic Role Matrix and Automatic Tree Reports, as they also use it. Keep me posted on your developments!
| Anhang | Größe |
|---|---|
| LDIFStruct_part3.zip | 309.71 KB |
Disclaimer: As with everything else at Cool Solutions, this content is definitely not supported by Novell (so don't even think of calling Support if you try something and it blows up).
It was contributed by a community member and is published "as is." It seems to have worked for at least one person, and might work for you. But please be sure to test, test, test before you do anything drastic with it.
Related Articles
User Comments
- Be the first to comment! To leave a comment you need to Login or Register
- 5329 reads
0