Article

rkalfane's picture
article
Reads:

7829

Score:
4
4
3
 
Comments:

0

Build your own Scripts to Manipulate LDIF Exports - Part 2

Author Info

23 January 2008 - 11:37am
Submitted by: rkalfane

Author Blog
Author Profile

Tags

Tags Map

(View Disclaimer)

Build your own Scripts to Manipulate LDIF Exports - Part 2

Table of Contents

Introduction
Using LDIFStruct_part2.zip
Scripting basics for each language
        Php
        Perl
        Vim
        BeanShell
        Python
        VBScript
        Java
        Mono/C#
Summary


Introduction

In the first article, Build your own scripts to manipulate LDIF exports - Part 1, you learned how to use associative arrays and lists in different languages. In this part, you'll learn about the following scripting basic features:

  • Parsing command-line arguments
  • Listing files in the current directory
  • reading file lines in an array for further manipulations
  • Using regular expressions to analyze content

These features will be explained using the following languages:

Python and VBScript have been added, as these two can be very useful in your projects. VB Scripts can run on almost any Windows machine without installing anything. Like the other languages we've covered, Python is cross-platform and extremely powerful when it comes to manipulating lists and arrays. Don't hesitate to skip directly to the languages of your interest!


Using LDIFStruct_part2.zip

Here is the content of the archive LDIFStruct_part2.zip:

./LDIFStruct
  +- /Php
  |  +- /test_basics.php*
  |  +- /file1.ldif
  +- /Vim
  |  +- /test_basics.vim*
  |  +- /test_basics.sh*
  |  +- /file1.ldif
  +- /Python
  |  +- /test_basics.py*
  |  +- /file1.ldif
  +- /VBScript
  |  +- /test_basics.vbs*
  |  +- /file1.ldif
  +- /Java
  |  +- /TestBasics.jar
  |  +- /TestBasics.java
  |  +- /TestBasics.class
  |  +- /TestBasics$1.class
  |  +- /test_basics.sh*
  |  +- /file1.ldif
  +- /Mono
  |  +- /TestBasics.cs
  |  +- /TestBasics.exe*
  |  +- /test_basics.sh*
  |  +- /file1.ldif
  +- /Perl
  |  +- /test_basics.pl*
  |  +- /file1.ldif
  +- /BeanShell
     +- /test_basics.bsh*
     +- /test_basics.sh*
     +- /file1.ldif
     +- /bsh-2.0b4.jar

All files test_basics.* are executable scripts that demonstrat the basics for each language. Most scripts will list the command-line arguments provided, if any, and will list some of the files in the current directory. They will also open the file file1.ldif for reading and use basic regular expressions on the file lines to analyze the content (find DN and attributes).


Scripting Basics for Each Language

The results of most of the test_basics.* scripts demonstrating scripting basics should look like this:

 1 Parsing the command-line arguments
 2 There is 5 arguments from the command-line:
 3 Argument #1 = ./test_basics.php
 4 Argument #2 = one
 5 Argument #3 = two
 6 Argument #4 = three
 7 Argument #5 = four
 8 
 9 Listing filtered files
10 test_basics.php
11 test_arrays.php
12 Found 2 .php file(s) in current directory
13 
14 Reading file file1.ldif and join LDIF multi-line
15 Found line beginning with space, joining with last line
16 Found line beginning with space, joining with last line
17 Found line beginning with space, joining with last line
18 Found line beginning with space, joining with last line
19 Found line beginning with space, joining with last line
20 Found line beginning with space, joining with last line
21 Found line beginning with space, joining with last line
22 Found line beginning with space, joining with last line
23 Found line beginning with space, joining with last line
24 Found line beginning with space, joining with last line
25 Found line beginning with space, joining with last line
26 Found line beginning with space, joining with last line
27 Found line beginning with space, joining with last line
28 Found line beginning with space, joining with last line
29 Found line beginning with space, joining with last line
30 
31 Result LDIF data
32 # LDIF export of groups
33 version: 1
34 
35 dn: cn=group3,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
36 changetype: add 
37 cn: group3
38 owner: cn=member5,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
39 manager: cn=member7,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
40 member: cn=member3,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
41 member: cn=member1,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
42 member: cn=member1,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
43 member: cn=member2,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
44 objectClass: groupOfNames
45 
46 dn: cn=group2,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
47 changetype: add
48 cn: group2
49 objectClass: groupOfNames
50 
51 dn: cn=group1,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
52 changetype: add
53 cn: group1
54 owner: cn=member4,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
55 manager: cn=member1,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
56 member: cn=member2,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
57 member: cn=member3,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
58 member: cn=member3,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
59 member: cn=member3,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
60 objectClass: groupOfNames
61 
62 Looping through all lines and detect DN and attributes
63 DN found: cn=group3,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
64 Attribute found: cn = group3
65 Attribute found: owner = cn=member5,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
66 Attribute found: manager = cn=member7,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
67 Attribute found: member = cn=member3,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
68 Attribute found: member = cn=member1,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
69 Attribute found: member = cn=member1,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
70 Attribute found: member = cn=member2,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
71 Attribute found: objectClass = groupOfNames
72 DN found: cn=group2,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
73 Attribute found: cn = group2
74 Attribute found: objectClass = groupOfNames
75 DN found: cn=group1,ou=groups,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
76 Attribute found: cn = group1
77 Attribute found: owner = cn=member4,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
78 Attribute found: manager = cn=member1,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
79 Attribute found: member = cn=member2,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
80 Attribute found: member = cn=member3,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
81 Attribute found: member = cn=member3,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
82 Attribute found: member = cn=member3,ou=users,ou=auth,ou=orgunit3,ou=orgunit2,ou=orgunit1,o=organization,dc=data
83 Attribute found: objectClass = groupOfNames
Note:Forthe Vim script, there is no test for command-line arguments and files listing. For Python, there are two more tests demonstrating special lists manipulation (concept called List Comprehension). For some languages, there could be one more command-line argument containing the name of the script itself.

Php

Parsing Command-line Arguments

With Php scripts, you can retrieve the number of arguments provided in the command-line with the variable $argc. The values of the arguments are stored in a normal array $argv. You can display this information using the code sample shown below. Looping through the values is done exactly the same way as in the first article):

 1 #!/usr/bin/php5
 2 <?
 3 // Parsing command-line arguments
 4 print "Parsing the command-line arguments\n";
 5 print "There is ".$argc." arguments from the command-line:\n";
 6 $i = 1;
 7 foreach( $argv as $value )
 8 {
 9   print "Argument #$i = $value\n";
10   $i++;
11 }
12 print "\n";

Listing Filtered Files in the Current Directory

Here are the main functions to use in Php to open a directory, list files and close it:

  • opendir( directory ) that returns a "handler" on the directory
  • readdir( handler ) that lists all files in the given directory
  • closedir( handler ) that closes the handler

In addition to retrieving all files, you can also filter only *.php files using a regular expression. The method used here is preg_match( pattern, text ) - that will check whether given text matches a pattern. The pattern '/.*\.php' checks that the filename ends with ".php". Note that the dot in the extension ".php" is backslahed, as it is a special regular expression character. There are more details about regular expressions in the last part of the code.

14 // Listing filtered files in the current directory
15 print "Listing filtered files\n";
16 $dir = ".";
17 $ext = "php";
18 if ($dh = opendir($dir))
19 {
20   $count = 0;
21   while ( ($file = readdir($dh)) !== false) 
22   {
23     if ( preg_match( '/.*\.'.$ext.'$/', $file ) )
24     {
25       echo $file."\n";
26       $count++;
27     }
28   }
29   closedir($dh);
30 }
31 print "Found $count .$ext file(s) in current directory\n";
32 print "\n";

Reading a File in a List and Joining LDIF Multi-files

To read a file in an array with Php you need to open it, read lines using a buffer, and store the lines in the array. The simplest code to achieve this is the following:

 1 // Reading file in a list
 2 print "Reading file file1.ldif\n";
 3 $data = array();
 4 $fd = fopen ( "file1.ldif", "r");
 5 while (!feof ($fd))
 6 {
 7   $buffer = fgets($fd, 4096);
 8   array_push( $data, $buffer );
 9 }
10 fclose ($fd);

In the provided example script, the script opens a LDIF file containing multi-lines starting with a space. The following code checks whether the line starts with a space; if it does, the current line is joined with the previous line, and the lines are stored in an array. The script uses an array_pop() function to get and remove the last element. It is joined with the current line and put it back in the array with the function array_push(). Php provides an impressive number of functions related to arrays, check the documentation!

34 // Reading file in a list and join LDIF multi-file
35 print "Reading file file1.ldif and join LDIF multi-line\n";
36 $data = array();
37 $fd = fopen ( "file1.ldif", "r");
38 while (!feof ($fd))
39 {
40   $buffer = fgets($fd, 4096);
41   // Check if line begins with a space, join with last line  
42   if ( preg_match( '/^ .*/', $buffer ) )
43   {
44     // Join LDIF multi-lines
45     print "Found line beginning with space, joining with last line\n";
46     // Pop last line and remove line return
47     $lastline = array_pop( $data );
48     $lastline = preg_replace( '/\n/', '', $lastline );
49     // Join result with current line without space
50     $currentline = preg_replace( '/^ /', '', $buffer );
51     array_push( $data, $lastline.$currentline );
52     
53   }
54   // Normal line
55   else
56   {
57     array_push( $data, $buffer );
58   }
59 }
60 fclose ($fd);
61 print "\n";

Displaying a Result with Joined Lines

To display the result of previous operations, you can simply loop through the data array:

63 // Display result with joined lines
64 print "Result LDIF data\n";
65 foreach ( $data as $line )
66 {
67   print $line;
68 }
69 print "\n";

Looping through Lines and Analyzing Content

Here are the most important Php functions for regular expressions (for most of them, there are also optional arguments available):

  • preg_match( pattern, text ) that performs a regular expression match
  • preg_replace( pattern, replacement, text ) that performs a regular expression search and replace. Use $1, $2, ... to use replacement parameters.
  • preg_grep( pattern, array ) that returns an array with elements matching the pattern from the given array
  • preg_split( pattern, string ) that splits a string into an array using the given pattern
71 // Loop through lines and analyze content
72 print "Looping through all lines and detect DN and attributes\n";
73 foreach ( $data as $line )
74 {
75   if ( preg_match( '/^(.*?): (.*)\n/', $line ) > 0 )
76   {
77     $elem1 = preg_replace( '/^(.*?): (.*)\n/', '\1', $line );
78     $elem2 = preg_replace( '/^(.*?): (.*)\n/', '\2', $line );
79     if ( $elem1 == "dn" )
80     {
81       print "DN found: ".$elem2."\n";
82     }
83     elseif ( preg_match( '/^version: |changetype: |^modify: |^add: |^replace: |^delete: /', $line ) == 0 )
84     {
85       print "Attribute found: ".$elem1." = ".$elem2."\n";
86     }
87   }
88 }
89 ?>

Once in the ./LDIFStruct/Php directory, you can call the script test_basics.php with arguments:

/LDIFStruct/Php> ./test_basics.php arg1 arg2 arg3 arg4

The script must be executable. If it is not, you can make it executable by using the following command in Linux:

/LDIFStruct/Php> chmod +x test_basics.php

If directly calling test_basics.php doesn't work, you can also call the php executable with test_basics.php as argument. For instance, on my Linux box with Php5 installed:

/LDIFStruct/Php> /usr/bin/php5 test_basics.php arg1 arg2 arg3 arg4

or on a Windows box, if php.exe is in the execution path:

C:\LDIFStruct\Php\> php.exe test_basics.php arg1 arg2 arg3 arg4

If you want to store the result in a text file to better read it, redirect the output:

/LDIFStruct/Php> ./test_basics.php arg1 arg2 arg3 arg4 > test_basics.log

Perl

Parsing Command-line Arguments

With Perl scripts, you can retrieve the number of arguments provided in the command-line directly by printing the variable @ARGV. The different values of the arguments are stored in the array @ARGV. You can display this information using the following code (looping through the values is done in exactly the same way as in the first article):

 1 #!/usr/bin/perl -w
 2 
 3 # Parsing command-line arguments
 4 print "Parsing the command-line arguments\n";
 5 print "There is ".@ARGV." arguments from the command-line:\n";
 6 my $count = 1;
 7 foreach my $value ( @ARGV )
 8 {
 9   print "Argument #".$count." = ".$value."\n";
10   $count++;
11 }
12 print "\n";

Listing Filtered Files in the Current Directory

Here are the main functions to use in Perl to open a directory, list files, and close it (very close to Php):

  • opendir( handler, directory ) that opens a "handler" on the directory
  • readdir( handler ) that lists all files in the given directory
  • closedir( handler ) that closes the handler

In addition to retrieving all files, you can also filter only *.pl files using a regular expression. The method used here is grep( pattern, array ) that returns an array with elements matching the pattern from the given files array. The pattern "\\.pl\$' checks that file names end with ".pl". Note that the dot and the dollar sign in the extension ".pl" are backslashed, as they are special regular expression characters. There are more details about regular expressions in the last part of the code.

14 # Listing filtered files in the current directory
15 print "Listing filtered files\n";
16 my $dir = ".";
17 my $ext = "pl";
18 opendir(DIR, $dir);
19 my $pattern = "\\.pl\$";
20 my @files = grep(/$pattern/,readdir(DIR));
21 closedir(DIR);
22 foreach $file (@files) 
23 {
24  print $file."\n";
25 }
26 print "Found ".@files." .".$ext." file(s) in current directory\n";
27 print "\n";

Reading files in a List and Joining LDIF multi-file

To read a file in an array with Perl you need to open it, and read lines directly in the array. The simpliest code to achieve this is the following (pretty short!):

 1 # Reading file in a list
 2 print "Reading file file1.ldif\n";
 3 open( DAT, "file1.ldif" ) || die( "Could not open file!" );
 4 my @data = <DAT>;
 5 close( DAT );

In the example below, the script opens a LDIF file containing multi-lines starting with a space. The following code checks if the line starts with a space; if it does, the current line is joined with the previous line. Lines are stored in an array, and the script uses a pop() function to get and remove the last element. It is joined with the current line and put it back in the array with the push() function. Perl provides a number of functions related to arrays, check the documentation!

29 # Reading file in a list
30 print "Reading file file1.ldif\n";
31 open( DAT, "file1.ldif" ) || die( "Could not open file!" );
32 my @raw_data = <DAT>;
33 close( DAT );
34 my @data = ();
35 foreach my $line ( @raw_data )
36 {
37   # Check if line begins with a space, join with last line   
38   if ( $line =~ /^ (.*)/ )
39   {
40     # Join LDIF multi-lines
41     print "Found line beginning with space, joining with last line\n";
42     # Pop last line and remove line return
43     $lastline = pop( @data );
44     $lastline =~ s/\n//;
45     # Join result with current line without space
46     $currentline = $1;
47     push( @data, $lastline.$currentline."\n" );
48   }
49   # Normal line
50   else
51   {
52     push( @data, $line );
53   }
54 }
55 print "\n";

Displaying a Result with Joined Lines (Perl)

To display the result of the previous operations, you can simply loop through the data array:

57 # Display result with joined lines
58 print "Result LDIF data\n";
59 foreach my $line ( @data )
60 {
61   print $line;
62 }
63 print "\n";

Looping through Lines and Analyzing Content

Here are the most common ways to use regular expressions in Perl (check this page for details):

  • Test for matches inline, e.g., "$line =~ m/mypattern/;"
  • Extract groups inline, e.g., "$line =~ m/whatever1(mypattern1)whatever2/;" - automatically using the variable '$1' in your code that will be the string-matching "mypattern1". This is used in the following code to match typical LDIF line "attribute: value" and extract the attribute name and the value at the same time! Unbeatable...
  • Search and replace strings using a pattern inline, e.g., "$line =~ tr/Joe/John/;" - replacing "Joe" by "John" in the variable $line

Here is the code used in the sample to detect a DN, attributes, and their values. Note that the simple operation "$line =~ /^(.*?): (.*)\n/" matches for an LDIF line and extracts attributes and values at the same time.

65 # Loop through lines and analyse content
66 print "Looping through all lines and detect DN and attributes\n";
67 foreach my $line ( @data )
68 {
69   if ( $line =~ /^(.*?): (.*)\n/ )
70   {
71     if ( $1 eq "dn" )
72     {
73       print "DN found: ".$2."\n";
74     }
75     elsif ( !( $1 =~ /^version|^changetype|^modify|^add|^replace|^delete/ ) )
76     {
77       print "Attribute found: ".$1." = ".$2."\n";
78     }
79   }
80 }

Once in the ./LDIFStruct/Perl directory, you can call the test_basics.pl script:

/LDIFStruct/Perl> ./test_basics.pl arg1 arg2 arg3 arg4

The script must be executable. If it is not, you can make it executable by using the following command in Linux:

/LDIFStruct/Perl> chmod +x test_basics.pl

If you want to store the result in a text file to better read it, redirect the output:

/LDIFStruct/Perl> ./test_basics.pl arg1 arg2 arg3 arg4 > test_basics.log

Vim

Note:To use lists and dictionaries in Vim, you will need Vim version 7 and higher.

Reading a File in a List and Joining LDIF Multi-files

When a file is opened in Vim, you can easily retrieve one line or an array of lines from the current buffer using the function getline(). The simpliest code to achieve this is the following (one line):

 1 let lines = getline( 1, '$' )

In the example below, the script opens a LDIF file containing multi-lines starting with a space. The following code checks if the line starts with a space; if it does, the current line is joined with the previous line. Lines are stored in an array, and the script accesses the last line using s:data[-1]. That line is joined with the current line and put it back in the array using s:data[-1]. Vim provides a number of functions related to arrays; check the documentation (:help script or :help eval).

 1 #!/usr/local/bin/vim -S
 2 
 3 let basedir = '.' 
 4 let file1 = 'file1.ldif'
 5 
 6 "Load first file in an array
 7 echo 'Loading file file1.ldif in first structure'
 8 "execute 'tabnew'
 9 execute 'e '.basedir.'/'.file1
10 
11 let s:data = []
12 let s:count = 0
13 while s:count < line('$')
14   let s:count = s:count + 1
15   let s:line = getline(s:count)
16   if matchstr(s:line,'^ .*') != ''
17     "Join LDIF multi-lines
18     let s:currentline = substitute(s:line,'^ \(.*\)','\1','')
19     let s:prevline = s:data[-1]
20     let s:data[-1] = s:prevline.s:currentline
21   else
22     call add(s:data, s:line)
23   endif
24 endwhile

Displaying a Result with Joined Lines

To display the result of the previous operations, you can simply loop through the data array:

26 "Display data
27 let s:index = 0
28 while s:index < len(s:data)
29   let s:line = s:data[s:index]
30   let s:index = s:index + 1
31   echo s:line
32 endwhile
33 echo " "

Looping through Lines and Analyzing Content

Here are the most important Vim functions for regular expressions:

  • match(expression, pattern) - tests the expression using the pattern
  • matchlist(expression, pattern) - tests the expression and get submatches using the pattern, returning a list
  • substitute(expression, pattern, substitution, flags) - does a search and replace using the pattern and the substitution string
37 "Analyse LDIF content
38 let s:index = 0
39 while s:index < len(s:data)
40   let s:line = s:data[s:index]
41   let s:index = s:index + 1
42   " Test if we have a typical LDIF line 'xxx: yyy'
43   let s:elements = matchlist(s:line,'^\(.\{-}\): \(.*\)')
44   if s:elements != []
45     "Check if current line contains a new dn
46     if s:elements[ 1 ] == 'dn'
47       echo "DN found: ".s:elements[ 2 ]
48     "Check if current line contains a new attribute
49     elseif matchstr(s:elements[ 1 ],'^version\|^changetype\|^modify\|^add\|^replace\|^delete') == ''
50       echo "Attribute found: ".s:elements[ 1 ]." = ".s:elements[ 2 ]
51     endif
52   endif
53 endwhile

Once in the ./LDIFStruct/Vim directory, you can call the test_basics.vim or test_basics.sh scripts:

/LDIFStruct/Vim> ./test_basics.vim
or
/LDIFStruct/Vim> ./test_basics.sh

The scripts must be executable. If they are not, you can make them executable by using the following command in Linux:

/LDIFStruct/Vim> chmod +x test_basics.vim
/LDIFStruct/Vim> chmod +x test_basics.sh

The result will be displayed in a Vim buffer.

BeanShell

Parsing Command-line Arguments

With BeanShell scripts, you can retrieve the values of the arguments in the bsh.args array . The number of arguments is then bsh.args.length. You can display these information using the following code (looping through the values is done in exactly the same way as in the first article):

 1 #!/etc/alternatives/java bsh.Interpreter
 2 
 3 // Parsing command-line arguments
 4 print( "Parsing the command-line arguments" );
 5 print( "There is " + bsh.args.length + " arguments from the command-line:" );
 6 i = 1;
 7 for( value : bsh.args )
 8 {
 9   print( "Argument #" + i + " = " + value );
10   i++;
11 }
12 print( "" );

Listing Filtered Files in the Current Directory

Given a path in BeanShell, or Java, you can build a File object and then use the list() method to get the list of files in the directory. You can also call this method using a filter object that tells which files to list. In our case, we can build such a filter by testing if the end of the filename is ".bsh".

14 // Listing filtered files in the current directory
15 print( "Listing filtered files" );
16 dir = ".";
17 ext = "bsh";
18 filter = new FilenameFilter() { public boolean accept(File dir, String name) { return name.endsWith("."+ext); } };
19 currentdir = new File( dir );
20 files = currentdir.list( filter );
21 for( filename : files )
22 {
23   print( filename );
24 }
25 print( "Found " + files.length + " ." + ext + " file(s) in current directory" );
26 print( "" );

You can use a regular expression instead of a filter when looping through the files list:

 1 // Listing filtered files in the current directory
 2 print( "Listing filtered files" );
 3 dir = ".";
 4 ext = "bsh";
 5 currentdir = new File( dir );
 6 files = currentdir.list( filter );
 7 count = 0
 8 for( filename : files )
 9 {
10   if ( filename.matches( ".*\\." + ext + "$" ) ) 
11   {
12     print( filename );
13     count++;
14 }
15 print( "Found " + count + " ." + ext + " file(s) in current directory" );
16 print( "" );

Reading a File in a List and Joining LDIF Multi-files

Reading a file is a bit more complex with the Bean Shell than with other languages. Here you need to open a Reader on the file and use a BufferedReader to be able to read the file line by line. Here is a code sample that reads all lines in the file file1.ldif and stores them in an ArrayList object:

 1 // Reading file in a list
 2 print( "Reading file file1.ldif" );
 3 line = "";
 4 data = new ArrayList();
 5 try
 6 {  
 7   in = new BufferedReader(new FileReader("file1.ldif"));
 8   if (!in.ready())
 9     throw new IOException();
10   while ((line = in.readLine()) != null)
11   {
12     data.add(line);
13   }
14   in.close();
15 }
16 catch (IOException e)
17 {
18   print("Error");
19   return map;
20 }

In the provided example script, the script opens a LDIF file containing multi-lines starting with a space. The following code checks if the line starts with a space; if it does, the current line is joined with the previous line. Lines are stored in an array, and the script accesses the last line using data.get( data.size() - 1 ). It is then joined with the current line and put it back in the array using data.set( data.size() - 1, newline ):

28 // Reading file in a list and join LDIF multi-file
29 print( "Reading file file1.ldif and join LDIF multi-line" );
30 line = "";
31 data = new ArrayList();
32 try
33 {  
34   in = new BufferedReader(new FileReader("file1.ldif"));
35   if (!in.ready())
36     throw new IOException();
37   while ((line = in.readLine()) != null)
38   {
39     // Check if line begins with a space, join with last line 
40     if ( line.matches( "^ .*" ) )
41     {
42       print( "Found line beginning with space, joining with last line" );
43       // Join LDIF multi-lines
44       data.set( data.size() - 1, data.get( data.size() - 1 ) + line.substring( 1 ) );
45     }
46     else
47     {
48       // Normal line
49       data.add(line);
50     }
51   }
52   in.close();
53 }
54 catch (IOException e)
55 {
56   print("Error");
57   return map;
58 }

Displaying the Result with Joined Lines

To display the result of the previous operations, you can simply loop through the data array:

60 // Display result with joined lines
61 print( "Result LDIF data" );
62 for( line : data )
63 {
64   print( line );
65 }
66 print( "" );

Looping through Lines and Analyse Content

Here are the most important BeanShell / Java functions available on Strings for regular expressions:

  • java.lang.String.matches(pattern) which tests the string using the pattern
  • java.lang.String.replaceAll(pattern,replacement) which does a search and replace using the pattern and the replacement string
  • java.lang.String.split(pattern) which splits a string into an array of strings using the given pattern
68 //Loop through lines and analyse content
69 print( "Looping through all lines and detect DN and attributes" );
70 for( line : data )
71 {
72   if ( line.matches( "^.*?: .*" ) )
73   {
74     elem1 = line.replaceAll( "^(.*?): (.*)", "$1" );
75     elem2 = line.replaceAll( "^(.*?): (.*)", "$2" );
76 
77     if ( elem1.equals( "dn" ) )
78     {
79       print( "DN found: " + elem2 );
80     }
81     else if ( !line.matches( "^version: .*|^changetype: .*|^modify: .*|^add: .*|^replace: .*|^delete: .*" ) )
82     {
83       print( "Attribute found: " + elem1 + " = " + elem2 );
84     }
85   }
86 }

You can also use dedicated java.util.regex.Pattern and java.util.regex.Matcher classes to easily extract strings using a pattern. Here are the methods you can use:

  • java.util.regex.Pattern.compile( pattern ) - compiles a pattern to use later on
  • java.util.regex.Pattern.matcher( line ) - creates a Matcher object to apply the pattern on the line
  • java.util.regex.Matcher.find() - executes the pattern on the line and extracts the strings
  • java.util.regex.Matcher.group( index ) - accesses the extracted strings
 1 p = Pattern.compile( "^(.*?): (.*)" );
 2 m = p.matcher( line );
 3 if ( m.find() )
 4 {
 5   elem1 = m.group(1);
 6   elem2 = m.group(2);
 7   ...
 8   ...
 9 }

Once in the ./LDIFStruct/BeanShell directory, you can call the test_basics.sh or test_basics.bsh scripts :

/LDIFStruct/BeanShell> ./test_basics.sh arg1 arg2 arg3 arg4
or
/LDIFStruct/BeanShell> ./test_basics.bsh arg1 arg2 arg3 arg4

The scripts must be executable. If they are not, you can make them executable by using the following command in Linux:

/LDIFStruct/BeanShell> chmod +x test_basics.sh
/LDIFStruct/BeanShell> chmod +x test_basics.bsh

The test_basics.sh script call a Java command, with the file bsh-2.0b4.jar in the classpath and the file test-basics.bsh as argument.

 1 #!/bin/sh
 2 
 3 export CLASSPATH=bsh-2.0b4.jar
 4 ./test_basics.bsh $*

You can call the test_basics.bsh script directly if you edit it to set your correct java binary path and if this file bsh-2.0b4.jar is in the classpath. You can add this file in your classpath on Linux by editing your .bashrc file for instance and add the following line:

export CLASSPATH=/path/to/your/file/bsh-2.0b4.jar

If you want to store the result in a text file to better read it, redirect the output:

/LDIFStruct/BeanShell> ./test_basics.sh arg1 arg2 arg3 arg4 > test_basics.log
or
/LDIFStruct/BeanShell> ./test_basics.bsh arg1 arg2 arg3 arg4 > test_basics.log

Python

Parsing command-line arguments

With Python scripts, you can retrieve the values of the arguments in the array sys.argv. The number of arguments is then len( sys.argv ). Don't forget to import the package "sys" to use this. You can display these information using the following code (looping through the values is done in exactly the same way as in the first article):

 1 #!/usr/bin/python
 2 
 3 import os
 4 import re
 5 import sys
 6 
 7 # Parsing command-line arguments
 8 print "Parsing the command-line arguments v1"
 9 print "There is %d arguments from the command-line:" % (len( sys.argv ),)
10 i = 1
11 for arg in sys.argv:
12   print "Argument #%d = %s" % (i,arg)
13   i = i + 1
14 print ""

Listing Filtered Files in the Current Directory

Listing files in Python can be done the classic way by getting files list in an array, then cycling through the array to check if the extension is ".py":

21 # Listing filtered files in the current directory
22 print "Listing filtered files v1"
23 dir = "."
24 ext = "py"
25 total = 0
26 files = os.listdir( dir )
27 for filename in files:
28   if re.search( '^.*\.' + ext + '$', filename ):
29     print filename
30     total = total + 1
31 print "Found %d .%s file(s) in current directory" % ( total, ext )
32 print ""

Python also has a very powerful feature called List Comprehension, making it very natural and easy to build lists. For instance, reading a file in an array would be:

 1 import os
 2 files = [f for f in os.listdir( "." )]

or:

 1 import os
 2 files = map( lambda f: f, os.listdir( "." ) )

You can also build a filtered list by adding a condition:

 1 import os
 2 files = [f for f in os.listdir( "." ) if re.search('^.*\.py$',f)]

Here is the code from the test_basics.py file that uses this feature:

34 print "Listing filtered files v2 (list comprehension)"
35 dir = "."
36 ext = "py"
37 files = [f for f in os.listdir( dir ) if re.search('^.*\.' + ext + '$',f) ]
38 print "\n".join( files )
39 print "Found %d .%s file(s) in current directory" % ( len( files ), ext )
40 print ""

Reading A File in a List and Joining LDIF Multi-files

In the example below, the script opens an LDIF file containing multi-lines starting with a space. The following code checks if the line starts with a space; if it does, the current line is joined with the previous line. Lines are stored in an array, and the script accesses the last line using data.pop(). Then it join its with the current line and puts it back in the array using data.append(newline):

42 # Reading file in a list and join LDIF multi-file
43 print "Reading file file1.ldif and join LDIF multi-line"
44 file = open("file1.ldif")
45 data = []
46 try:
47   for line in file:
48     match = re.match( '^ (.*\n)', line )
49     # Check if line begins with a space, join with last line
50     if match:
51       print "Found line beginning with space, joining with last line"
52       # Pop last line and remove line return
53       lastline = re.sub( '(.*)\n', '\\1', data.pop() )
54       # Join result with current line without space
55       newline = lastline + match.group(1)
56       data.append( newline )
57     # Normal line
58     else:
59       data.append( line )
60 finally:
61   print ""
62   file.close()

Displaying a Result with Joined Lines

To display the result of the previous operations, you can simply loop through the data array:

64 # Display result with joined lines
65 print "Result LDIF data"
66 print "".join( data )

Looping through Lines and Analyzing Content

Here are the most important Python functions available for regular expressions:

  • match = re.match( pattern, line ) which tests the line using the pattern, it is also extracting the different blocks in parenthesis which can be then accesed using match.group( 1 ), match.group( 2 ), etc.
  • re.split( pattern, string ) that splits a string into a list using the given pattern
  • re.sub( pattern, replacement, string ) that performs a regular expression search and replace.
68 # Loop through lines and analyze content
69 print "Looping through all lines and detect DN and attributes"
70 for line in data:
71   # Test if we have a typical LDIF line "xxx: yyy"
72   match = re.match( '^(.*?): (.*)\n', line )
73   if match:
74     # If xxx is a dn
75     if match.group(1) == "dn":
76       print "DN found: %s" % ( match.group(2), )
77     # If xxx is an attribute meaning no changetype or add or replace or delete
78     elif not re.match( 'version|changetype|add|replace|delete', match.group(1) ):
79       print "Attribute found: %s = %s" % ( match.group(1), match.group(2) )

Once in the ./LDIFStruct/Python directory, you can call the script test_basics.py:

/LDIFStruct/Python> ./test_basics.py arg1 arg2 arg3 arg4

The script must be executable. If it is not, you can make it executable by using the following command in Linux:

/LDIFStruct/Python> chmod +x test_basics.py

If you want to store the result in a text file to better read it, redirect the output:

/LDIFStruct/Python> ./test_basics.py arg1 arg2 arg3 arg4 > test_basics.log

VBScript

Parsing Command-line Arguments

With VB scripts, you can retrieve the values of the arguments in the WScript.Arguments array. The number of arguments is then WScript.Arguments.Count. You can display this information using the following code (looping through the values is done in exactly the same way as in the first article):

 1 ' Parsing command-line arguments
 2 WScript.Echo "Parsing the command-line arguments"
 3 Dim count
 4 set objArgs = WScript.Arguments
 5 WScript.Echo "There is " & WScript.Arguments.Count & " arguments from the command-line:"
 6 count = 1
 7 for each strArg in objArgs
 8   WScript.Echo "Argument #" & count & " = " & strArg
 9   count = count + 1
10 next
11 WScript.Echo ""

Listing Filtered Files in the Current Directory

Listing files with VB is done using FileSystemObject class:

13 ' Listing filtered files in the current directory
14 WScript.Echo "Listing filtered files"
15 Dim fso: Set fso = CreateObject("Scripting.FileSystemObject")
16 Path = "."
17 Ext = "vbs"
18 Dim Parent, Filter
19 Parent = Path
20 Dim n: n = 0
21 Dim Folder: Set Folder = fso.GetFolder(Parent)
22 Dim Files: Set Files = Folder.Files
23 Dim File
24 For Each File In Files
25   If Right(File,4) = "." & Ext Then
26     Set objFile = fso.GetFile( File )
27     Wscript.Echo "File name: " & fso.GetFileName(objFile)
28     n = n + 1
29   End If
30 Next
31 WScript.Echo "Found " & n & " ." & Ext & " file(s) in current directory"
32 WScript.Echo ""

Reading a File in a List and Joining LDIF Multi-files

Reading from a file also uses the same FileSystemObject class. The simplest code to read a file in a list would be the following (note the use of ArrayList object for dynamic arrays):

 1 ' Reading file in a list
 2 WScript.Echo "Reading file file1.ldif"
 3 Set data = CreateObject( "System.Collections.ArrayList" )
 4 Const ForReading = 1
 5 Set objFSO = CreateObject("Scripting.FileSystemObject")
 6 Set objTextFile = objFSO.OpenTextFile("file1.ldif", ForReading)
 7 Do Until objTextFile.AtEndOfStream
 8   strNextLine = objTextFile.Readline
 9   data.Add strNextLine
10 Loop

In the example below, the script opens a LDIF file containing multi-lines starting with a space. The following code checks if the line starts with a space; oif it does, the current line is joined with the previous line. Lines are stored in an array, and the script accesses the last line using data.Item( data.Count - 1 ). Then it joins it with the current line and puts it back in the array using the same data.Item( data.Count - 1 ):

34 ' Reading file in a list and join LDIF multi-file
35 WScript.Echo "Reading file file1.ldif and join LDIF multi-line"
36 Set data = CreateObject( "System.Collections.ArrayList" )
37 Const ForReading = 1
38 Set objFSO = CreateObject("Scripting.FileSystemObject")
39 Set objTextFile = objFSO.OpenTextFile("file1.ldif", ForReading)
40 Do Until objTextFile.AtEndOfStream
41   strNextLine = objTextFile.Readline
42   Set myRegExp = New RegExp
43   myRegExp.Pattern = "^ (.*)"
44   If myRegExp.Test( strNextLine ) Then
45     WScript.Echo "Found line beginning with space, joining with last line"
46     data.Item( data.Count - 1 ) = data.Item( data.Count - 1 ) & Mid( strNextLine, 2 )
47   Else   
48     data.Add strNextLine
49   End If
50 Loop
51 WScript.Echo ""

Displaying a Result with Joined Lines

To display the result of the previous operations, you can simply loop through the data array:

53 ' Display result with joined lines
54 WScript.Echo "Result LDIF data"
55 For Each line In data
56   Wscript.Echo line
57 Next
58 WScript.Echo ""

Looping through Lines and Analyzing Content

Here are the main functions that can be used for regular expressions in VB when creating a new RegExp object:

  • myRegExp.Pattern( pattern ) that compiles a regular expression for later use
  • myRegExp.Test( line ) that tests the defined expression on the line
  • Set myMatches = myRegExp.Execute( line ) that performs a search using the pattern defined and extract the strings that you can then accessed using myMatches(0).SubMatches(0), myMatches(0).SubMatches(1), etc.
60 ' Loop through lines and analyse content
61 WScript.Echo "Looping through all lines and detect DN and attributes"
62 For Each line In data
63   ' Test if we have a typical LDIF line "xxx: yyy"
64   Set myRegExp = New RegExp
65   myRegExp.Pattern = "^(.*?): (.*)"
66   Set myMatches = myRegExp.Execute(line)
67   If myMatches.Count > 0 Then
68    ' If xxx is a dn
69     Set myMatch = myMatches(0)
70     If myMatch.SubMatches(0) = "dn" Then
71       WScript.Echo "DN found: " & myMatch.SubMatches(1)
72    ' If xxx is an attribute meaning no changetype or add or replace or delete
73     Else
74       Set myRegExp = New RegExp
75       myRegExp.Pattern = "^version|changetype|add|replace|delete"
76 
77       If not myRegExp.Test( myMatch.SubMatches(0) ) Then
78         WScript.Echo "Attribute found: " & myMatch.SubMatches(0) & " = " & myMatch.SubMatches(1)
79       End If
80     End If
81   End If
82 Next

Once in the ./LDIFStruct/VBScript directory, you can call the test_basics.vbs script :

C:\LDIFStruct\VBScript\> cscript test_basics.vbs arg1 arg2 arg3 arg4

If you want to store the result in a text file to better read it, redirect the output:

C:\LDIFStruct\VBScript\> cscript test_basics.vbs arg1 arg2 arg3 arg4 > test_basics.log

Java

The sample code below starts with a class definition, importing the different packages to use:

 1 import java.util.*;
 2 import java.io.*;
 3 
 4 public class TestBasics
 5 {
 6   public static void main( String[] args )
 7   {
 8     ...
 9     ...

Parsing Command-line Arguments

With Java, you can retrieve the values of the arguments in the array args passed in the main method. The number of arguments is then args.length. You can display this information using the following code. Looping through the values is done in exactly the same way as in the first article):

 8     // Parsing command-line arguments
 9     System.out.println( "Parsing the command-line arguments" );
10     System.out.println( "There is " + args.length + " arguments from the command-line:" );
11     int i = 1;
12     for( String value : args )
13     {
14       System.out.println( "Argument #" + i + " = " + value );
15       i++;
16     }
17     System.out.println( "" );

Listing Filtered Files in the Current Directory

Given a path in Java, you can build a File object and then use the list() method to get the list of files in the directory. You can also call this method using a filter object that will tell which files to list. In our case, we can build such filter by testing if the end of the filename is ".java". As with BeanShell, you can also loop through all the files and use a regular expression to filter java files.

19     // Listing filtered files in the current directory
20     System.out.println( "Listing filtered files" );
21     String dir = ".";
22     final String ext = "java";
23     FilenameFilter filter = new FilenameFilter() { public boolean accept(File dir, String name) { return name.endsWith("."+ext); } };
24     File currentdir = new File( dir );
25     String[] files = currentdir.list( filter );
26     for( String filename : files )
27     {
28       System.out.println( filename );
29     }
30     System.out.println( "Found " + files.length + " ." + ext + " file(s) in current directory" );
31     System.out.println( "" );

Reading a File in a List and Joining LDIF Multi-files

Reading a file is a bit more complex in Java than in other languages. Here you need to open a Reader for the file and use a BufferedReader to read the file line by line. Below is a code sample that reads all lines in file1.ldif and stores them in an ArrayList object:

33     // Reading file in a list
34     System.out.println( "Reading file file1.ldif" );
35     String fileline = "";
36     ArrayList<String> data = new ArrayList<String>();
37     try
38     {  
39       BufferedReader in = new BufferedReader(new FileReader("file1.ldif"));
40       if (!in.ready())
41         throw new IOException();
42       while ((fileline = in.readLine()) != null)
43       {
52         // Normal line
53         data.add(fileline);
55       }
56       in.close();
57     }
58     catch (IOException e)
59     {
60       System.out.println("Error");
61       System.exit(1);
62     }

In the example script below, the script opens an LDIF file containing multi-lines starting with a space. The following code checks if the line starts with a space; if it does, the current line is joined with the previous line. Lines are stored in an array, and the script accesses the last line using data.get( data.size() - 1 ). Then it is joined with the current line and put it back in the array using data.set( data.size() - 1 ):

33     // Reading file in a list and join LDIF multi-file
34     System.out.println( "Reading file file1.ldif and join LDIF multi-line" );
35     String fileline = "";
36     ArrayList<String> data = new ArrayList<String>();
37     try
38     {  
39       BufferedReader in = new BufferedReader(new FileReader("file1.ldif"));
40       if (!in.ready())
41         throw new IOException();
42       while ((fileline = in.readLine()) != null)
43       {
44         // Check if line begins with a space, join with last line   
45         if ( fileline.matches( "^ .*" ) )
46         {
47           // Join LDIF multi-lines
48           data.set( data.size() - 1, data.get( data.size() - 1 ) + fileline.substring( 1 ) );
49         }
50         else
51         {
52           // Normal line
53           data.add(fileline);
54         }
55       }
56       in.close();
57     }
58     catch (IOException e)
59     {
60       System.out.println("Error");
61       System.exit(1);
62     }

Displaying a Result with Joined Lines

To display the result of the previous operations, you can simply loop through the data array:

64     // Display result with joined lines
65     System.out.println( "Result LDIF data" );
66     for( String line : data )
67     {
68       System.out.println( line );
69     }
70     System.out.println( "" );

Looping through Lines and Analyzing Content

Here are the most important Java functions available on Strings for regular expressions:

  • java.lang.String.matches(pattern) which tests the string using the pattern
  • java.lang.String.replaceAll(pattern,replacement) which does a search and replace using the pattern and the replacement string
  • java.lang.String.split(pattern) which splits a string into an array of strings using the given pattern
72     //Loop through lines and analyze content
73     System.out.println( "Looping through all lines and detect DN and attributes" );
74     for( String line : data )
75     {
76       if ( line.matches( "^.*?: .*" ) )
77       {
78         System.out.println( "Found line beginning with space, joining with last line" );
79         String elem1 = line.replaceAll( "^(.*?): (.*)", "$1" );
80         String elem2 = line.replaceAll( "^(.*?): (.*)", "$2" );
81 
82         if ( elem1.equals( "dn" ) )
83         {
84           System.out.println( "DN found: " + elem2 );
85         }
86         else if ( !line.matches( "^version: .*|^changetype: .*|^modify: .*|^add: .*|^replace: .*|^delete: .*" ) )
87         {
88           System.out.println( "Attribute found: " + elem1 + " = " + elem2 );
89         }
90       }
91     }

You can also use dedicated java.util.regex.Pattern and java.util.regex.Matcher classes to easily extract strings using a pattern. Here are the methods you can use:

  • java.util.regex.Pattern.compile( pattern ) which compile a pattern to use later on
  • java.util.regex.Pattern.matcher( line ) which create a Matcher object to apply the pattern on the line
  • java.util.regex.Matcher.find() which execute the pattern on the line and extract the strings
  • java.util.regex.Matcher.group( index ) which access the extracted strings

Once in the ./LDIFStruct/Java directory, you can call the test_basics.sh script :

/LDIFStruct/Java> ./test_basics.sh arg1 arg2 arg3 arg4

The script must be executable. If it is not, you can make it executable by using the following command in Linux:

/LDIFStruct/Java> chmod +x test_basics.sh

If you want to store the result in a text file to better read it, redirect the output:

/LDIFStruct/Java> ./test_basics.sh arg1 arg2 arg3 arg4 > test_arrays.log

If you change the TestBasics.java source code, here are the commands to use to compile and rebuild the TestBasics.jar file (you can then execute the test_arrays.sh script):

/LDIFStruct/Java> javac TestBasics.java
/LDIFStruct/Java> jar cvf TestBasics.jar TestBasics*.class
added manifest
adding: TestBasics.class(in = 2826) (out= 1385)(deflated 50%)

Mono/C#

The sample code starts by a class definition as the following, importing the different packages to use:

 1 using System;
 2 using System.IO;
 3 using System.Collections;
 4 using System.Text.RegularExpressions;
 5 
 6 class TestBasics
 7 {
 8   static void Main( string[] args )
 9   {
10     ...
11     ...

Parsing Command-line Arguments

With Mono/C#, you can retrieve the values of the arguments in the array args passed in the main method. The number of arguments is then args.Length. (Note the "L" uppercase for Length, which is different from Java.) You can display this information using the following code. Looping through the values is done in exactly the same way as in the first article):

10     // Parsing command-line arguments
11     Console.WriteLine( "Parsing the command-line arguments" );
12     Console.WriteLine( "There is " + args.Length + " arguments from the command-line:" );
13     int i = 1;
14     foreach( string value in args )
15     {
16       Console.WriteLine( "Argument #" + i + " = " + value );
17       i++;
18     }
19     Console.WriteLine( "" );

Listing Filtered Files in the Current Directory

To list files using Mono, you can use the DirectoryInfo object. The method GetFiles from this object can use a filter with a "*" wildcard, such as "*.cs".

21     // Listing filtered files in the current directory
22     Console.WriteLine( "Listing filtered files" );
23     string dir = ".";
24     string ext = "cs";
25     DirectoryInfo di = new DirectoryInfo( dir );
26     FileInfo[] rgFiles = di.GetFiles("*."+ext);
27     foreach(FileInfo fi in rgFiles)
28     {
29       Console.WriteLine( fi.Name );     
30     }
31     Console.WriteLine( "Found " + rgFiles.Length + " ." + ext + " file(s) in current directory" );
32     Console.WriteLine( "" );

Reading a File in a List and Joining LDIF Multi-files

Reading a file in Mono/C# is very similar to Java. You must use a FileStream and a StreamReader class to read lines. The simplest code to read a file in a list would be the following:

 1     // Reading file in a list
 2     Console.WriteLine( "Reading file file1.ldif" );
 3     string fileline;
 4     ArrayList data = new ArrayList();
 5     StreamReader reader = new StreamReader(new FileStream("file1.ldif", FileMode.Open));
 6     while((fileline = reader.ReadLine()) != null)
 7     {
 8       data.Add( fileline );
 9     }
10     reader.Close();

In the example below, the script opens a LDIF file containing multi-lines starting with a space. The following code checks if the line starts with a space; if it does, the current line is joined with the previous line. Lines are stored in an array, and the script accesses the last line using data[ data.Count - 1 ]. Then it joins it with the current line and puts it back in the array using the same data[ data.Count - 1 ]. Note: this is similar but simplier than in VB):

34     // Reading file in a list and join LDIF multi-file
35     Console.WriteLine( "Reading file file1.ldif and join LDIF multi-line" );
36     string fileline;
37     ArrayList data = new ArrayList();
38     StreamReader reader = new StreamReader(new FileStream("file1.ldif", FileMode.Open));
39     while((fileline = reader.ReadLine()) != null)
40     {
41       // Check if line begins with a space, join with last line  
42       if ( Regex.IsMatch( fileline, "^ " ) )
43       {
44         // Join LDIF multi-lines
45         data[ data.Count - 1 ] = data[ data.Count - 1 ] + fileline.Substring( 1 );
46       }
47       else
48       {
49         data.Add( fileline );
50       }
51     }
52     reader.Close();

Displaying a Result with Joined Lines

To display the result of the previous operations, you can simply loop through the data array:

54     // Display result with joined lines
55     Console.WriteLine( "Result LDIF data" );
56     foreach( string line in data )
57     {
58       Console.WriteLine( line );
59     }
60     Console.WriteLine( "" );

Looping through Lines and Analyzing Content

Here are the most important functions you can use for regular expressions in Mono/C#:

  • Regex.IsMatch( line, pattern ) which test the line using the given pattern.
  • Regex.Replace( line, pattern, replacement ) which does a search and replace.
62     //Loop through lines and analyse content
63     Console.WriteLine( "Looping through all lines and detect DN and attributes" );
64     foreach( string line in data )
65     {
66       if ( Regex.IsMatch( line, "^.*?: .*" ) )
67       {
68         string elem1 = Regex.Replace( line, "^(.*?): (.*)", "$1" );     
69         string elem2 = Regex.Replace( line, "^(.*?): (.*)", "$2" );
70 
71         if ( elem1.Equals( "dn" ) )
72         {
73           Console.WriteLine( "DN found: " + elem2 );
74         }
75         else if ( !Regex.IsMatch( line, "^changetype: .*|^modify: .*|^add: .*|^replace: .*|^delete: .*|^version: .*" ) )
76         {
77           Console.WriteLine( "Attribute found: " + elem1 + " = " + elem2 );
78         }
79       }
80     }

Once in the ./LDIFStruct/Mono directory, you can call the test_basics.sh script:

/LDIFStruct/Mono> ./test_basics.sh arg1 arg2 arg3 arg4

The script must be executable. If it is not, you can make it executable by using the following command in Linux:

/LDIFStruct/Mono> chmod +x test_basics.sh

If you want to store the result in a text file to better read it, redirect the output:

/LDIFStruct/Mono> ./test_basics.sh arg1 arg2 arg3 arg4 > test_arrays.log

If you change the TestBasics.cs source code, here is the command to use to compile and rebuild the TestArrays.exe file (you can then execute the test_basics.sh script):

/LDIFStruct/Java> mcs TestBasics.cs


Summary

This AppNote is an introduction on how to handle command-line parameters, list files in a directory, read lines from a file, and use regular expressions to analyze the content. Using these features with arrays and associative arrays presented in the first article, you can build very complex scripts.

Remember: you can go directly to the section of your favorite language and use copy/paste to quickly start building your own scripts! For each language I have presented one possible solution for each operation, but don't forget that there is always more than one way!

In the next and last part article, you will learn how to build the "LDIFStruct" library by combining all the features together. You'll also learn how to use the structures created from LDIF exports to generate XML, merged LDIF, CSV exports, SQL statements, etc.

As you can see in this part, we are now able to open a LDIF file and analyze it to retrieve DNs and attributes and their values. Now we need to build our dictionaries/lists structure as explained in the first article to reach our goal.

AttachmentSize
LDIFStruct_part2.zip276.59 KB

Disclaimer: As with everything else at Cool Solutions, this content is definitely not supported by Novell (so don't even think of calling Support if you try something and it blows up).

It was contributed by a community member and is published "as is." It seems to have worked for at least one person, and might work for you. But please be sure to test, test, test before you do anything drastic with it.



Related Articles


User Comments

© 2012 Novell