Monthly Archives: September 2009

Tivoli Directory Integrator – Part 6: Updating An External LDAP Server From Domino

By Marie Scott and Thomas “Duffbert” Duff

In this installment, we’re going to delve a little deeper into Tivoli Directory Integrator by demonstrating how to update an external LDAP server with Domino Directory [Duffbert – Notes Address Book aka NAB or names.nsf using LDAP – for us developer types] data using a TDI LDAP connector.  This particular AssemblyLine will also stretch our TDI skills by require some JavaScript to parse the Domino data into the format needed by the external LDAP server. 

Before we begin this process, we need to have a Domino server that is running the LDAP server task.  We’re using an LDAP type connection for Domino for our demonstration.  We only need reader access to the Person documents in the Domino Directory because we are not updating fields in those documents.   Likewise, we are going to update records in the external LDAP directory, so we need the appropriate login credentials and update authority in the LDAP tree.  We are not deleting attributes, so we do not need delete authority in this example.

 In this particular AssemblyLine, our goal is to update some Domino-related attributes in our external LDAP server.  We’re including these values in our external LDAP entries for each person, so that Domino can be later configured to use the external LDAP as the source for HTTP, IMAP, and Sametime authentication. [Marie – Domino admins out there will note that Directory Assistance will be used on the Domino side for authentication…this is a handy trick!  Duffbert – yeah, yeah… get to the good stuff!] The attributes have already been added to the schema on the external LDAP server, as they did not previously exist.  What is important to note is that we need a unique key to match the records from Domino to LDAP.  In our example we use unid (which is an LDAP default identifier) and an employee id number.   The AssemblyLIne does not require two keys to link data attributes, but in this case we are using two, and as you’ll see the AssemblyLine will allow you to use multiple keys.

The attributes we’re going to work with are the following:

  • $dn
  • DominoUserAbbrev
  • DominoUserName
  • displayname
  • mailfile
  • mailserver
  • sametimeserver
  • employeeid
  • uid 

    To get started we create a new TDI Configuration.  

    tdipart06-11new

    Next we create a new AssemblyLine.

    tdipart06.22new

    Now we’ll create a new Connector called DominoLDAP in “Interator” mode.

    tdipart06-33new

    So now we have a Domino LDAP connector.  Next, we’ll create our connector to our external LDAP server.  This will work exactly the same as the one we just created, but we’ll call it EDIRLDAP and set it up as an “Add Only” Connector.

    tdipart06-44new

    So how do we connect to our Domino server via the LDAP connector?  We connect using the LDAP url and port of our Domino LDAP server, with a userid and HTTP password that has access to read the Domino Directory. [Marie – we can also set up an SSL connection by selecting that option.]

    tdipart06-5b

    We’re also limiting the LDAP search filter by using the objectclass=dominoperson.  This will give us only those attributes in the dominoperson schema.

    We use the same method to connect to the external LDAP by connecting to the specific LDAP server and limiting the search base and search filter based on what we’re going to update in the LDAP schema.

    Once we set up our login information, we’ll want to make sure we can actually “talk” to the servers via TDI.  To do this, we use the Establish Connection button (the little plug)

    tdipart06-5c

    If all the information is correct in the previous step, we’ll see that a “Connection Established message.

    tdipart06-5d

    Now, we can click the “Read the Next Entry” button (the little triangle) to cycle through the entries in the actual LDAP directory.  This will allow us to view real “live” data to confirm the formatting.

    So to recap – we’ve created a new AssemblyLine, created two new connectors, and tested the connections.    What’s next?  Now we want to add the connectors to the appropriate places in our AssemblyLine. We can do this by dragging and dropping the “DominoLDAP” connector over to our AssemblyLine Feed indicator.  Add the External LDAP connector – as our AssemblyLine “flow” by the same method.

    tdipart06-2c

    How do we pick fields we want to work with?  A couple of things to keep in mind – the Feed connector will have an INPUT tab and your Flow connector will have an OUTPUT tab.  These tabs are where we work with selecting our data fields also known as Work Entries:

    tdipart06-55new

    Okay, these next steps are a little hairy so pay close attention.  On the FEED/INPUT Map – we have WORK Attributes.   These are the select group of data fields from our Domino Directory.  Some of these Work Attributes actually exist in the schema of the Domino Directory, and we can use them as is, so we can drag and drop them from the Schema View over to our Work Attribute list. 

    tdipart06-66new

    Once we’ve selected a Work attribute, say for example “givenname”, if we know that we’re going to use that value as is, we can enable that in the Attribute Map settings (the Attribute Map is displayed when a Work Attribute is selected individually). So we’ve selected the Work Attribute by name in the Attribute Map as well as selected “Enabled” and type as “Simple”.

    tdipart06-77new

    Now we do need to do some parsing, as the format from Domino does not match that of the LDAP results as we showed in the table above.  The method for making those changes involves pulling the data from the feed and making the change to the work items, and then parsing the data when it is output.  We’re going to use JavaScript to perform these operations.

    Let’s start with one of the Work Attributes – mailfile.  Because the data in the mailfile field from the Domino Directory needs to be parsed, we use JavaScript to change how the data appears.  We do this in the Attribute Map Setting by selecting Advanced (JavaScript).

    tdipart06-88new

    We also have some Work Attributes that don’t exist in the Domino Directory, but they will need to map to fields in our external LDAP directory.  So we click the Add New Attribute button.  In this example we are going to add a Work Attribute called  “DominoUserName” that will use data from the Work Attribute called “$dn”, and we’re going to do some Advanced (JavaScript).

    tdipart06-99new

    Once we have completed updating both the Work Attributes (Feed) and Connector Attributes (Flow), we should have the following fields with the appropriate coding:

    $dn

    Work Attribute

    No advanced mapping;Enabled

    DominoUserAbbrev

    WorkAttribute

    temp=conn.getString(“displayname”);

    ret.value=temp.toLowerCase();

    DominoUserName

    WorkAttribute

    temp =conn.getString(“$dn).replace(“,”,”/”);

    ret.value=temp.toLowerCase();

    givenname

    WorkAttribute

    No advanced mapping;Enabled

    Mail

    WorkAttribute

    No advanced mapping;Enabled

    Mailfile

    WorkAttribute

    ret.value=conn.getString(“mailfile”).replace(“\\”,”/”) + “.nsf”;

    MailServer

    WorkAttribute

    ret.value=conn.getString(“mailserver”).replace(“,”,”/”);

    Sametimeserver

    WorkAttribute

    ret.value=conn.getString(“sametimeserver”).replace(“,”,”/”);

    ssn

    WorkAttribute

    No advanced mapping;Enabled

    uid

    WorkAttribute

    No advanced mapping;Enabled

    $dn

    Connector Attribute

    ret.value=”uid=” +work.getString(“uid”) + “,ou=people,org=domain”

    DominoUserAbbrev

    Connector

    Attribute

     

    ret.value=work.getString(“DominoUserAbbrev”);

    DominoUserName

    Connector

    Attribute

    ret.value=work.getString(“DominoUserName”);

    givenname

    Connector

    Attribute

    No advanced mapping;Enabled

    Mail

    Connector

    Attribute

    No advanced mapping;Enabled

    Mailfile

    Connector

    Attribute

     

    ret.value=work.getString(“MailFile”);

    MailServer

    Connector

    Attribute

     

    ret.value=work.getString(“MailServer”);

    Sametimeserver

    Connector

    Attribute

     

    ret.value=work.getString(“sametimeserver”);

    eduPersonID

    Connector

    Attribute

    No advanced mapping;Enabled

    uid

    Connector

    Attribute

    No advanced mapping;Enabled

    So how do we “join” the two separate sets of data?  We link them via our keys and the Link Criteria tab on the AssemblyLine Flow attribute.  As we said we’re going to make sure that our entries in the External LDAP server are updated only when the unid and $unid are the same AND when the employeeid is the same.  So we’ll link those attributes on the Link Criteria.

    On the Flow Object – EDIRLDAP – go to the Link Criteria tab.  Click on the Add New Link Criteria icon (the little chain) and select “uid” as the connector attribute, “equals” as the operator, and “$uid” as the value.

    tdipart06-3b

    tdipart06-3a

    uid is the value in the Domino Directory, and $uid is the value in the external LDAP Directory.

    So we go ahead and create a similar link criteria for “employeeid” as the connector attribute, “equals” as the operator, and the attribute that contains the employeeid in the external LDAP directory.  [Marie: Attribute and value names don’t have to match when establishing them as link criteria.]

    Now that we’ve set up linking, and customized our Work/Connector Attributes, our Work Entry view should look something like this:

    tdipart06-55new

    Whew!  Now would be a perfect time to save the AssemblyLine configuration as we certainly wouldn’t want to have to rebuild all those connections.  And a word of caution before we proceed to the next step… As with any system, you want to make sure that you have a test environment to work with before you put this into the LIVE REAL WORLD!  TDI is very powerful, and could easily wipe data from the feed or the flow ends of the AssemblyLine.  So please don’t take our names in vain and say that we didn’t warn you!

    So we’re ready to start our AssemblyLine and see how we’ve done.  We’re going to select Standard as our Run Mode and will click the start button to complete.

    tdipart06-new3

    A tool that may come in handy when you’re working with LDAP entries on different types of servers is an LDAP browser, so you can review your data.  One we’ve used before is the Softerra LDAP Browser (they have a free version!).

    Here’s one of our diagrams to display what we’ve done:

    tdildapexample

    Using this example, you can now synchronize your Domino LDAP directories to all your other external LDAP directories without having to invest in an expensive tool to do so.  That should make you very popular with your boss, and may be a leverage point when you ask him for permission to go to Lotusphere!

  • No, no! Say it’s not so!

    Sometimes you can’t always believe what you read.  For example, The Commonwealth Times, the student newspaper at Virginia Commonwealth University, posted the following article. If you take the content for what it says, it does sound like the entire university is moving to Google.  However, this is not the case.  Then another blogger posted this blog.  And in the context of his blog he made some assumptions which really weren’t accurate.  I responded to those assumptions in kind.

    As a result, I’ve learned several valuable lessons.  Make sure that content posted on the web is accurate.  Secondly, don’t assume that the author is an expert in the field of the content material or that they have vetted their sources.  With today’s web commentary — that is not always the case.  And for some reason Google seems to hit a hot button with everyone these days — whether it be with migration from one email system or another or Google vulnerabilities, etc. etc., we all like to find something newsworthy to post about Google.  Well maybe we need to step back and make sure the information we have is accurate.  I include myself in that “we” as I have blogged about Google as well.  I would also caution those blogging as to the reasons why a company moves to Google.  With firsthand experience, I can say that the decision making process to move to any new messaging platform is complicated and not as straightforward as it may seem.  So the collective “we” should not shoot those managers who have had to make those difficult decisions or those technologists who have had to implement those decisions in the foot, because who knows “we” might be next.

    A Lotusphere Anxiety Attack

    Hi my name is Marie, and I’ve had a Lotusphere anxiety attack.  Well, truth be told I’ve had several of them.

    For the first year in a long time, it may not be a given that I will be able to attend Lotusphere.  Now for those of you who have only attended one or two Lotuspheres this may seem rather silly.  Having an anxiety attack? Come on now.  But given that Lotusphere 2010 would be my twelveth Lotusphere, it truly has become an annual event.  My year would not be complete without the “family reunion” in Orlando.  As the Virginia State government budget is in crisis right now, all travel expenditures are being reviewed. My fingers are crossed that my trip will be approved.

    Secondly, for the first time I am working on some abstracts to submit as a speaker.  While I have fantastic possible co-speakers, I’m still so, so nervous about being on the other side of the stage if accepted.  It sounds like there is a lot of great material going to be submitted this year, so we should have the best Lotusphere yet!

    Thirdly, Lotusphere registration is later than usual.  And don’t we all like to have our registration in hand by now?  Some of you may not remember those years when you had to be poised the ready on the first day because registration filled up.  Again…why should I be nervous?  I just am!!!

    In sum, in case you can’t tell — I am looking forward to Lotusphere this year, probably more so than any other year.  As a new blogger, I can’t wait to meet all the folks I’ve met through my blog and through Twitter.  And I’m also anticipating all the great content we have to look forward to with 8.5.1 and beyond!

    January 17, 2010 here we come! :-)

    Tivoli Directory Integrator – Part 5: Loading A Notes Database from a CSV File

    By Marie Scott and Thomas “Duffbert” Duff

    So now that you’ve got TDI set up and running on your workstation, the question becomes how to make it do something useful.In this installment, we’ll take a CSV file containing some data and use TDI to load a Notes database on a local Notes server. This is a very basic example, but it’s something that all Notes developers and administrators have had to do numerous times. And our hope is that it will start to acclimate you with the basic steps to set up a TDI AssemblyLine.


    So first, start up TDI as well as your Domino server and Notes client.  In this example, we have a CSV file of book titles that Duffbert has read, and we want to import that into a Notes database (so he can keep track of what he’s already read). 

    TDIPart05Image01

    TDIPart05Image02

    Nothing fancy here… We’re just looking to get the basics in connectivity down.


    So begin by starting Tivoli Directory Integrator.  When it first loads, you’ll see a blank command screen (which is the Java background task loading), then the TDI GUI will load.  To begin, you’ll want to create a new configuration. That’s done by using the File > New menu option, which will present you with a dialog screen.For this example we’ll call this configuration blogExample01.xml.

    TDIPart05Image03

    That will create your entire project you’ll use going forward.  The next thing you need to do once this is done is to make sure you have a particular entry in your java.library.path setting.  This is very important when you are running this in local Notes mode.  TDI will attempt to find the nlsxbe.dll file during the running of your import, so you have to be sure to have that path in the above java.library.path setting.  In this particular instance, we added c:\Program Files\IBM\Lotus\Domino.  Without that entry, you will receive a number of errors when the import starts to run.

    TDIPart05Image04

    Creating your new AssemblyLine comes next by using the Object > New AssemblyLine menu option.  We called ours CSVToNotes:

    TDIPart05Image05 
    With the AssemblyLine in place, we create our first connector to feed in data from the CSV file.  That is created with the Object > New Connector option, and you’ll want to choose the ibmdi.FileSystem connector type.  That’s the connector that can read a flat file from your workstation.  Give it a descriptive name (ReadCSVFile in our case) and put it in Iterator mode.  That means it will iterate through the entire file when used: 

    TDIPart05Image07 
    Our input connector feed is now created!  On the file path field, point to where you have the CSV file stored:

    TDIPart05Image08a

    The next step is where things get cool… if you click on the plug icon in the Input Map tab, TDI will automatically make a connection to your CSV file.Then if you click the forward icon next to it, TDI will iterate through the detail of the CSV file so that you can see the data you have stored out there.  These two steps will assure that you have connectivity to get to your data:

    TDIPart05Image09a

    The final step you need to do here is to highlight all four of the entries, and drag them over to the Work Attribute column next to the field names.  This is what will allow us to map the CSV columns to the Notes field names:

    TDIPart05Image10 
    Half your work is now done!  The next step is to create your other connector, the one that will allow you to write out to the Notes database.  This time, select the ibmdi.Notes connector wit h a mode of Add-Only.  We named our output connector OutputNotesFile:

    TDIPart05Image11

    Again, we’ll go to the connection tab under the Configuration tab and set up the connection information to the Domino server we have on the workstation.  This will obviously be specific to however you have your Domino server configured:

    TDIPart05Image12 
    At this point, we are simply mapping the CSV columns to the Notes form fields. Refer to the screen picture below:

    TDIPart05Image12 
    To get one or more columns from the CSV file, drag them from the Work Entry area on the lower left side into the Connector Attribute column.You can single-click on the Connector Attribute name in order to change it to the Notes field name.  In this case, the column names and the Notes field names were the same.  Then when you have the Connector Attribute field selected, make sure it’s connected to the right CSV column in the Select from work entry attributes area.

    For our Notes form, we do need to add one field that isn’t in there.  The Form field is needed so that Notes knows what form to launch the document with.  Here’s the example for that field:

    TDIPart05Image14 
    We clicked on the Create New Attribute icon at the top of the Connector Attribute column.  We then gave it a name of Form, selected a type of Advanced (JavaScript), and used the code snippet of ret.value=”frmBook”; to make sure that each document we create has a Form field in it.

    Go ahead and save your work to make sure nothing happens to it at this point!!

    Now it’s time to see the fruits ofyour labor.In the upper right corner of the TDI screen, you have your run options:

    TDIPart05Image15 
    Once you set it to run, you’ll see an output screen of runtime messages from TDI, and hopefully a newly populated Notes database!

    TDIPart05Image16

    TDIPart05Image17 
    Now that you have the basics set up, you can easily and quickly develop new connectors and AssemblyLines to feed in CSV files to Notes with little programming involved.  [Marie – Duffbert are you really saying that this might be useful to application developers?!]

    Here’s one of our diagram’s summarizing the process:

    tdiexample2

    And this is just the basics!  In our next article we’ll present a method for connecting to LDAP and we’ll get into more of the programming and hooks you can create with JavaScript that end up making TDI a far easier tool for Domino integration than having to manually set up new LotusScript jobs for every new import.


    Tivoli Directory Integrator: Part 4: Concepts & Vocabulary

    By Marie Scott and Thomas “Duffbert” Duff
     Before we get started with the nuts and bolts of Tivoli Directory Integrator, we figured it might be a good idea to introduce you to many of the concepts and vocabulary you’ll run into when learning about TDI.  That way, when you see terms like “assemblyline”, you‘ll know what is going on.  There’s nothing worse than trying to learn how to use a new technology when you don’t even know what the main terms mean! 
     

     So, you’ve got Domino directories, you’ve got Domino databases, you’ve got data that resides outside of Domino – in say Active Directory, or LDAP, or SQL.  Now what? Well to bring these systems together to merge or update data in either direction, we can use TDI. 
     

    To get started we’re going to build an AssemblyLine – just like a factory assembly line for manufacturing, the AssemblyLine in TDI is used to identify, move, transform, push, pull or synch data between various Data Sources.   
    A Data Source is the data system or group of data objects that you are going to connect via the AssemblyLine.   And a Connector acts as your means of setting up a logical connection to the Data SourceConnectors use authenticated processes like LDAP, DIIOP, JDBC, AD, etc., that provide dialog boxes for configuring exactly how the Data Source will be accessed. Within the Data Source are Entries. These are the data objects themselves.  And each entry has Attributes – or sub parts.  Attributes describe and or contain the data values from the various data sources that you’re linking together in the AssemblyLine.  Values are the objects that contain the actual data values that are being stored and transported through the AssemblyLine.

     
    These are batch or event-driven processes that handle the identification, routing, and transformation of data between data sources.  TDI can run any number of these AssemblyLines at any given time.  These AssemblyLines are what the developers and administrators of TDI will put together to accomplish their data needs. [Marie – I think of the AssemblyLine as similar to workflow in Domino.]

    Here’s a somewhat oversimplified diagram of the TDI process:
     

      
    Some more advanced terms include:
     

    EventHandlers – These are actions that are monitored by TDI, and that will trigger an AssemblyLine process.  This might be an LDAP change, a SOAP call, or a calendar event.

    Scripting – Scripting languages are used to create AssemblyLine steps that go beyond the built-in functionality provided in TDI.  JavaScript is the language supported in TDI for this function.
     
    Work Entry – an object created by TDI to move data throughout the AssemblyLine process, that is, the process that moves and/or transforms data from one data source to another.

    Delta Detection – When a data source has changed and the system needs to figure out what exactly has changed and what needs to be passed to other data sources that should remain synchronized. This is the first of three steps in the Delta operation.

    Delta Tagging – Once the Delta Detection has occurred, the data source needs to be annotated to flag exactly what has changed and how it has changed.  This helps the synchronization run efficiently.  It is the second part of the Delta operation.

    Delta Application – This reads the Delta Tagging (also known as Operation Codes) and makes the changes to synchronize the data sources. This is the final step in the Delta operation.

    Change Detection Connectors – used by TDI to find changes in standard data sources, such as LDAP, Active Directory, SQL databases, and Lotus Notes.

    Delta Engine – Used by TDI to compute changes for data sources that don’t flag updates in any other way.  This uses a Snapshot database to track prior values to determine if changes have occurred.

     
    Obviously, there’s far more to TDI than just this.  But if you understand these basic concepts, many things will start to drop into place when you start using TDI to move your own data around.
     
     
     

     

    Flash – Domino Does Not Change Time after DST Ends

    In case you missed the Support Flash — Domino 8.0.1 and 8.0.2 on some platforms (AIX, Linux, Solaris, i5/OS and z/OS) — is not changing to the appropriate time after Daylight Saving Time ends.  So check out technote 1326160 for appropriate SPRs, Fixpacks and related information!