Driver package build development lifecycle?


When we create a new driver we use base package, and all other customization is done through adding custom policies on the drivers, this gives us the faster way of developing test and verify, easy to export drive configuration files,  and making it lightweight, portable, (sharing common code functionality is achieved using policy linking),  and we use  GIT for versioning of our driver configurations files,

but development using packages experience compared to developing with driver configuration files is much confusion and feels unnecessary many "manual " steps when developing "package" based development using netiq designer.  looks like netiq idm package development has influenced by java developers but in java, you have proper build (a compiled code)  and release (CI/CD) pipelines processes which automates from "build" to "release" your solution to different environments which I don't see how this model fits within netiq designer and if it  has automted CI/CD pipeline process for driver package based development. all the driver package development is done in the form of  traditional Dirxml-Scripting "scripting" using a so-called "development" driver and adding its content to package. 

so some thoughts:

- When to use the package "build" option? (you have to deploy driver to identity vault before you can completely verify integration tests not just simulating xds documents within the designer simulation will verify it. since the build process is not required before you deploy a  driver, then why I need to actually create a build for my driver package? , if this is required for release of the driver package but then releasing driver package will make package  "read-only" anyways :), so not sure what is the purpose of the build here. 

- Deployment of driver again must have correctly linked attributes DirXML-pkgGUID to latest package build version, looks like a driver uses build version of driver package version to link package, not package version?


- Driver package management ?,  these driver package files (.jar) , how to manage them and their ownership and where to store them? one can not lock driver packages to a specific designer project and its ownership to a specific developer on a specific local machine. 

maybe someone shed some light on the driver package development process here? Tasks step by step, what others do for internal development for drivers using driver packages?

we are not vendors, we only develop drivers for our own company.






  • ,

    Hopefully the following rambling is of help in considering the use of packaging. I am not a full time developer, nor do I claim to be a packaging expert. I am someone who has become fairly comfortable with packaging over the past couple of years in my own lab and worked with various customers in looking at options and methods to take advantage of effective packaging processes. If you have not already reviewed the documentation for packages (Identity Manager Administration Guide - Developing Packages). Ironically, I found the documentation more helpful once I had figured out packaging, than when I first tried to understand the documentation. It seems there may be a chicken and an egg situation here (for myself at least) that was resolved after lots of personal experience. (Experience = "making lots of silly mistakes")

    To start with I keep separate Projects (if not separate workspaces) for the Production environment and the Test/Development environments I have worked with/in. If there are multiple environments and multiple folks involved, the Production workspace and project will be on a dedicated workstation/system that can be access controlled. Workspace separation also helps to avoid costly errors and untested deployments/changes being accidentally deployed to the production  environment(s).

    I am not sure where I picked up the following practice, though it has been very helpful for my own development process. Within the development project (Designer Modeler) I have at least three Vaults:

    • The first Vault (Live IDV) one is the representation of and has the connection to the test/development environment. This Vault object does not have packaging enabled at all. This Vault will have most, if not all, drivers in the development environment and is the one I use to deploy to a live environment for full testing. This Vault is often named the same as the test/development Tree it is connecting to.
    • The second Vault (Developer IDV) is where packaging is enabled and I do all the actual driver development and initial testing in Simulator (as much as I can). This Vault may have connection information to the live text/development environment, though I do not deploy from here. I may compare at times for reference and occasionally import from the live environment to get things like schema setup to match the live test environment.
    • The third Vault (Test IDV) is often empty and I will delete and recreate it a number of times to ensure it is a blank slate when testing the package deployment process. I can also use this to create out of the box drivers for comparisons/learning, etc. After a package has been developed (in the Developer IDV) and functional testing is completed (in the Live IDV), I will use this Vault (Test IDV) to verify the package has all the dependencies and initial configuration information configured correctly. I have surprised myself numerous times with little items overlooked or forgotten that are already deployed in test environments, but were not included in the package setup for deployment to a production environment. This allows me to capture those missing pieces and include them before the package is built and released.
    • There may be other Vaults for connected systems (eDir to eDir) off of the first Vault, but those are be specific to an environment and can be ignored in the general case.

    The development process I use also determines when a new build and version is created. So some thoughts on how my development cycle may be helpful here.

    Starting with a Base Package, I may use the delivered one out of the box, though I often clone it and modify it for the specific use case I have been working on. The main reasons for cloning and modifying the Base package is I can define my own Initial Settings, Dependencies and Configuration Wizard and have that specific to the target environment. Things like trace log file name, path, etc. are not included out of the box and can be included if desired. The Configuration Wizard can be setup to allow for the selection of the custom packages that will be developed as part of the whole solution. I can also define simple things like a custom icon for iManager and include that in the Base definition.

    Additional packages will be defined for general configuration as well as specific functions that can easily be grouped together. Maybe the policies and resources specific to Group management and synchronization are in one package, separate from the package that holds the policies and resources specific to user management. There will also be driver-set packages that contain common resources for multiple drivers as well as the overall environment. These could be mapping tables, global configuration settings, ecma script resources, etc. The driver-set packages can then also be defined as dependencies in the custom packages. An update to common (shared) resources only requires a change in that package when updating the live environments.

    Development of a specific custom package for a driver will be done in the Developer IDV. The same package being developed will also be added to the related driver in the Live IDV. As changes are made to the package and tested in the Developer IDV and then "Sync to Package", the driver in the Live IDV will get those updates. When I have reached a point that I need to test in a live system, vs testing in Simulator, I can simply deploy/compare from the Live IDV and then run my tests. This cycle of development, sync to package in the Developer IDV, deploy from the Live IDV and rinse and repeat as needed, continues until I have a stable working solution. I can also from time to time add the package(s) to a new/test driver in the Test IDV to make sure I am not missing some dependency in the package. I will go through these steps a few times at the end to make sure it is fully baked.

    Once I am comfortable that I have a good package, I will then Build and Release it as well as Publish it. Using either the build package or the Publish source directory, I can add the new package to the Production Workspace/Project and then deploy to production when allowed by the appropriate change practices. Should something go sidewise, it is a quick process to downgrade the package to the previous build and redeploy to the state prior. This is a huge plus for packaging in my mind - the ability to quickly roll-back if needed.

    The point that a build occurs could be the end of a development project, or simply a major milestone achieved that has a deliverable that can go to production. It could also be the point that the development is handed off to a QA team for further testing. Once released and sent off to the next step, I will create a new version that further development can be worked on and still have the option to roll back to that point in time in my Live IDV should some issue/bug be reported for further testing.

    There may be more than one package built and released at the same time if there is a group comprising a given solution. Changes in the future may only apply to a single package to address a given bug, or more than one if there is changes to broader functionality. As new packages are released from Micro Focus with new functionality, that can also be added to the solutions implemented so far.

    I should note that though I may be cloning out of the box drivers for customizing to meet a specific use case/need, common packages such as Password Management, Common Settings, ECMA script libraries are generally used as delivered.

    Going back to your questions/thoughts, I hope I have provided some ideas as to why and how packaging can be helpful, along with why a released build is a good thing.

    In terms of deployment of the driver, the package process and deployment cycle will maintain the correct values for the package attributes. As a package is upgraded or downgrading, these will be updated accordingly. Without a released build, you will not be able to roll forward or back to a specific point, or build, for testing, validation, roll-back, etc.. It also greatly complicates things if the project with the open build, that was deployed to your production environment, is lost or corrupted. Recreating the Project by importing from the Vault will have references to unavailable packages that cause further issues getting things working for further development once more. If you have released builds stored in the published location, they can be added to a clean project before importing from a Vault and then you are in good shape to proceed - should you ever need to rebuild from scratch.

    I would recommend that projects be checked into your version control system, as well as the various package builds. This way, others can check out the projects with the package development included and pick up on prior work for new versions of a given package. This avoids the potential issue of packages lost to specific developer or workstations should things break there. You could also unlock your own packages that are no longer associated to an available project if you need that option. (Unlocking-a-Designer-package-you-created)

    Hopefully the above is helpful. This is only one perspective and hopefully I have not implied that this is the right way. It is just a way that has worked well for myself and those I have worked with.



  •   Thanks for such good info:

    to simplify the driver package development process, would be it correct step by step as following for a driver package development for a specific custom solution?  (honestly, I have never seen or deployed factory standard NetIQ driver which fits our needs and our IT environments)


     1. Create a driver package, start from version 0.0.1

     2.  Choose a base package from NetIQ to head start for a development driver.   (Example RESTBase)

    3.   Add all the custom policy on development driver and use  "Add to package" from the development driver. 

    4.  Deploy driver to IDV (for testing and verification)

    5.  Once verified driver in IDV, Click  Build on Package

    6.  Deploy driver to IDV to have it correct package linkages (for build package)

    7.  Repeat process from 3-6 until the whole functionality is verified and tested

    8.  Finally, release and publish package and Re-Deploy driver to have latest package linking to be correct

    9 . New functionality on the driver will start with new package version 0.0.2

    10. Take copies of all packages somewhere in Subversion or Git for team sharing and in case of need.






  • Verified Answer

      I agree that I have never seen a factory standard driver fit the needs of any customer. I look at the out of the box solution as the default framework that provides the foundation to build/create a driver that meets the specific needs of the customer situation.

    I think your steps are good, though I would not get hung up on using your step 5 and 6 too early. Without a new build, each deploy of an open package will update the linkages if and as needed. You can repeat steps 3 and 4 a number of times, until you reach a point that is comfortable - based on your needs - to generate a build. For some that will be more often, while for others that will be at major milestones of development. Then you can also repeat steps 3 through 6 multiple times as part of your development. The key is to have some level of consistency and manageability for all involved in the process. Find the balance that works for you between having too many builds after each change and too few builds with lots of major changes in each cycle. I would suggest that a build would generally align with a key testing point and not with a development testing point.

    In terms of the version numbers used, that will be entirely up to you and your choices for managing the builds/releases. I usually use a new major version if there is a big change, initial release, or lots of time has passed since the driver was updated. (1.x.x to 2.x.x) That may also align somewhat with the underlying IDM version it is running on.

    The second number could be aligned with major development milestones and moving through from development to QA processing. (1.0.x to 1.1.x)

    The final number could be aligned with incremental testing changes or bug fixes. (1.0.0 to 1.0.1)

    The choice is yours to make in how you wish to structure your management of the development cycles. If you can establish some standard guidelines up front it will make easier to follow through.