Fernando Freitas wrote an awesome script that makes the User Application web service endpoint for Roles available as shell commands in bash. This is much more useful than you would think at first. After all, we can do them in a GUI, in SOAP UI, or other SOAP tools, what does bash give us?

Well if you are a shell command kind of guy or gal, it gives you a lot. Now you can use any other script framework, and just call out to these functions. As a simple example, I grabbed some sample scripts that take commands, processes them, and then executes a series of tasks, and then added in support to read input from a CSV file. Then adding in this bash shell extension, I was able to do User App role operations, in bulk, from a command line script in Linux. That turns out to be mighty useful.

For example, we had a customer who had thousands of roles, and needed to renamed 900 of them in a hurry. Sure we could have done it in the GUI with enough time. We THINK we might have been able to do it by generating LDIF files to change the two attributes that seem to hold the information, but we were not sure if that would break anything. Instead we needed to call the setRoleLocalizedString() API call twice, once to set the name, a second time to set the Description. Using the CSV reading script, we were able to call this 900 times (twice each time) with one command.

As you can see, this can be really useful. Now imagine you have 18,000 users you need to add to one or many roles. Lots of possible ways to do this, but having a script like this that can read input from a CSV sure makes it easy. You could for example use a Delimited Text driver (I recommend Stefaan Van Cauwenberge's Generic Text driver, it is free and way better than the NetIQ one) that has a policy to call the Add Role token for every line it reads in.

He has put a bunch of his plugins and tools available on his web site at:
http://vancauwenberge.info

You could do it in a Workflow that accepts text in a box, that is a list of DN's to add to a Role, and then call an Integration Activity to make the SOAP call. This has an issue that the User App will run out of memory as it loops pretty quickly, limiting the number of entries that can be processed in a single run.

However, using this script, you would be making many thousands of calls, but they queue up in order and get processed as fast as they can be processed. You can easily add a pause into the script to delay it if needed.

All this is nice to know, and if you read my series on how you might add a new function that Fernando missed, you will see how the script works and how you might extend it.

Series: Adding New Functions to the UA Bash extension

But I noticed in there, that although the script Fernando wrote is awesome, he took some steps to simplify things. For example, the case of createRole he did not add in support for owner, approval workflow, revoke workflow, quorum or some other possible settings. Now I understand the reasons why he did it this way. First off is simplicity, adding in the other options would have made his task much more difficult, and these are less common things to be set than others. However I think the real reason is that handling parameters that might be present, might be absent is a tiny bit trickier than the approach he took, and rather than solve that, he only handled the important ones. Also, it is pretty simple to add in such support if you need it.

You can see how he handles the absence of two values in the createRole function I want to talk about. On the command line parameter list the last two are category and correlation ID. Both of which can be absent, but not both absent on the same command. The mechanics by which he set it are not bad, but they do not scale to more than 2 missing parameters. It is really easy to handle the last parameter being absent or not, but any others and you get into frame shift issues. If #7 is missing, how do you know what #8, #9, and #10 are?

The simple fix seems to me just define a string value for empty fields. Thus early on, I would declare a variable NULL and assign it a simple string value which I will use as 'null' since I am lazy and unimaginative. In fact if you read the notes in the script, you will see a To Do section that says:

# TODO:
#  Decide on a good way to implement:
#  - findRoleByExampleWithOperator
#  - findSodByExample
#  - findSodByExampleWithOperator
# Current challenge is the sheer amount of different valid options, where all
#  of them can be used simultaneously. Barring some sort of common usage pattern  
#  the best way might be to use getopts and use parameter names, not only positions.
#
#  Figure how modifyRole works

The issue he identifies is similar to the one I have but his is a more complex case to handle. I am not sure how to do it the way he is suggesting so I will approach the part I do know how to accomplish.

First up, I will define my NULL variable in the section where all the basic variables are declared right after the debug setting.

export _NULL="null"

Then let us look at the rest of the function. For review, here is the version as it sits.

### Function: createRole
# Usage:
# createRole $username $password $rbpm_url $output_file $rolename $description $rolelevel $category $correlation_id
# if the $correlation_id is omitted, will call createRoleRequest, otherwise will call createRoleAidRequest
# if the category is omitted with use the value "default"
# Order of parameters is important, it is not possible to use the correlation_id and skip category
#
createRole()
{

USAGE="Function Usage:

createRole username password rbpm_url output_file rolename description rolelevel category correlation_id
 if the correlation_id is omitted, will call createRoleRequest, otherwise will call createRoleAidRequest
 if the category is omitted with use the value "default"
 Order of parameters is important, it is not possible to provide a correlation_id and skip category at the same time
 rbpm_url should be in the format:
  protocol://server:port/servicename
 for example:
  https://rbpm.lab.novell.com:8543/IDMProv";

if "X$_RBPM_SOAP_ROLE_DEBUG" = "Xtrue"
then
  dbgparams=$#
  dbgparam=1
  while [ "$dbgparam" -le "$dbgparams" ]
  do
    echo -n "Parameter " 
    echo -n \$$dbgparam
    echo -n " = "
    eval echo \$$dbgparam
    (( dbgparam   ))
  done
fi

# Initial Parameters check
if | -z "$2" || -z "$3" || -z "$4" || -z "$5" || -z "$6" || -z "$7"
then
  echo "$USAGE"
  return 1
fi

if -z "$8"
then
  CAT=default
else
  CAT="$8"
fi

if -z "$9"
then
  NOCID=true
else
  NOCID=false
  CID="$9"
fi

if "X$2" = "X-W"
then
  read -sp "Please enter the password for user $1: " SENHA
  echo
else
  SENHA=$2
fi

# Setup for the SOAP call
URL="${3}/role/service"

if "$NOCID" = "true"
then
  ACTION="SOAPAction: 'http://www.novell.com/role/service/createRole'"
  SOAPCALL=createRoleRequest
else
  ACTION="SOAPAction: 'http://www.novell.com/role/service/createRoleAid'"
  SOAPCALL=createRoleAidRequest
fi
CTYPE='Content-Type: text/xml;charset=UTF-8'

# Build SOAP XML envelope and call to be issued
POST="<soapenv:Envelope xmlns:soapenv='http://schemas.xmlsoap.org/soap/envelope/' xmlns:ser='http://www.novell.com/role/service'>\
<soapenv:Header/>\
<soapenv:Body>\
<ser:${SOAPCALL}>\
<ser:role>\
<ser:approvers/>\
<ser:container/>\
<ser:description>${6}</ser:description>\
<ser:entityKey/>\
<ser:name>${5}</ser:name>\
<ser:owners/>\
<ser:quorum/>\
<ser:requestDef/>\
<ser:revokeRequestDef/>\
<ser:roleCategoryKeys>\
<ser:categorykey>\
<ser:categoryKey>${CAT}</ser:categoryKey>\
</ser:categorykey>\
</ser:roleCategoryKeys>\
<ser:roleLevel>${7}</ser:roleLevel>\
<ser:systemRole>false</ser:systemRole>\
</ser:role>"

if "$NOCID" = "false" 
then
  POST="${POST}<ser:correlationId>${CID}</ser:correlationId>"
fi

POST="${POST}</ser:${SOAPCALL}>\
</soapenv:Body>\
</soapenv:Envelope>"

if "X$_RBPM_SOAP_ROLE_DEBUG" = "Xtrue"
then
  echo
  echo POST data:
  echo $POST
  echo
fi

# Issue the request
curl $_CURL_OPTIONS -k -u "$1:$SENHA" -H "$CTYPE" -H "$ACTION" -d "$POST" "$URL" -o "$4"
}

The issue that I see is in the section where the POST variable is set, the XML for the SOAP call is being built as a string, and some of the possible nodes are left empty.  Like:
	<ser:approvers/>
	<ser:container/>
	<ser:entityKey/>
	<ser:owners/>
	<ser:quorum/>
	<ser:requestDef/>
	<ser:revokeRequestDef/>

It seems like, you might want to script many cases where you need to set values like this. In order to add support for this we would need to see original SOAP example from the WSDL to get a feel for what the XML should look like. Then when we build the XML in the POST variable, like the way he handled correlation ID we can have a test for the specific parameter, and if the value passed in is null, we will send in the empty node of the XML, but if we have a real value passed in, we will append the proper XML. This way we can handle what is needed. Some SOAP API's are not constructed in a way that would make this approach easy (I am looking at YOU IMS Global, good grief, who came up with that idea? Yes, I am not a fan. To work with it, is to appreciate how painful it can be).

Let's start with owners, since that seems simple right? Just a DN? Well it turns out it can be many owner references. If you look at the XML in the WSDL it looks something like this:

    <ser:owners>
       <!--Zero or more repetitions:-->
       <ser:dnstring>
	  <ser:dn>?</ser:dn>
       </ser:dnstring>
    </ser:owners>

Zero or more repetitions means we can have 0, 1, or as many as we like. Now this is a twist I probably will not handle here. If I did, I would probably define a secondary delimiter character in my CSV parsing script, and thus take one parameter for owner, with its own internal separator. But that is more than I care to handle in this article so I will keep it simple and say only one owner this way. Probably need to work on modifyRole to add more than a single owner in this fashion.

Right now, the command line looks something like this, taken from the usage part of the script:
createRole username password rbpm_url output_file rolename description rolelevel category correlation_id

The last two are sort of handled as optionals now, so I would probably leave them as the last two so that the existing code that handles a value or absence of a value continues as it works now. Just remember to shift the values they look for, once the entire command line is finalized.

Thus for owner I would add it after rolelevel and before category, so it is the 8th command line parameter. Then in the section where the POST variable is getting set with XML, I would have a code block something like this:

if "X$8 = "X$_NULL"
then
  POST="${POST}<ser:owners/>"
else
  POST="${POST}<ser:owners>\
<ser:dnstring>\
<ser:dn>${8}</ser:dn>\
</ser:dnstring>\
</ser:owners>"  
fi

Then I would strip out the <ser:owners/> line from the script above.

Next, the field approvers is a very similar situation where you could have more than one, but at least one is better than none. Again, I would probably handle it the same was with owners with a secondary delimiter, but I am not going to include that here.

The SOAP XML for approvers looks like:

            <ser:approvers>
               <!--Zero or more repetitions:-->
               <ser:approver>
                  <ser:approverDN>?</ser:approverDN>
                  <ser:sequence>?</ser:sequence>
               </ser:approver>
            </ser:approvers>

So that means we add a field to the command line for approverDN and I guess we default to a sequence of 0 for the first one if we only do one. What I find difficult in SOAP usually is understanding what the underlying data should look like. Sometimes it is easy and obvious. Sometimes you can query to find examples and work from there. Recently I worked on the IMS Global LIS schema and it defines the query functionality as optional, so it seems no one bothered to implement queries. Which makes things really much harder than needed. You cannot query to see what the data looks like through the API to know what to send in, rather you have to figure it out or guess. Other times, you actually have to guess and try different options until you get it correct. But in this case, a sequence, seems likely to be ordering, so likely integers, and likely starting at 0 or 1. I think 0 is more likely, so I would start there.

if "X$9 = "X$_NULL"
then
  POST="${POST}<ser:approvers/>"
else
  POST="${POST}<ser:approvers>
<ser:approver>\
<ser:approverDN>${9}</ser:approverDN>\
<ser:sequence>0</ser:sequence>\
</ser:approver>\
</ser:approvers>\
fi

The next few fields are pretty simple single values, so lets handle them all as a series, since they are all basically the same. Wrap them in a test for null, and if it is, send in an empty node, and if not, send in the proper values based on the command line parameters.

<ser:container/>
<ser:entityKey/>
<ser:quorum/>
<ser:requestDef/>
<ser:revokeRequestDef/>

For container add it next.

if "X$10 = "X$_NULL"
then
  POST="${POST}<ser:container/>"
else
  POST="${POST}<ser:container>${10}</ser:container>"
fi

For quorum, which I do not understand the allowed values, add it as:

if "X$11 = "X$_NULL"
then
  POST="${POST}<ser:quorum/>"
else
  POST="${POST}<ser:quorum>${11}</ser:quorum>"
fi

For requestDef it is just an LDAP DN of the PRD, so that is an easy one. A long value, but an easy value to obtain.

if "X$12 = "X$_NULL"
then
  POST="${POST}<ser:requestDef/>"
else
  POST="${POST}<ser:requestDef>${12}</ser:requestDef>"
fi

Same issue for revokeRequestDef:

if "X$13 = "X$_NULL"
then
  POST="${POST}<ser:revokeRequestDef/>"
else
  POST="${POST}<ser:revokeRequestDef>${13}</ser:revokeRequestDef>"
fi

Finally entityKey, which I just recently found out is important in createResource, but I have no idea what it means in createRole, so I will support it, in case someone knows how to use it. In createResource, you can specify the entire DN of the object you are creating, which is apparently how you are supposed to make a Resource object in a resource container. The createRole function has a <ser:container> node for that, where you specify the container to place the Role in. But createResource does not, rather when you desire to create it in a container, use the entityKey value with a full LDAP DN of your target object.

if "X$14 = "X$_NULL"
then
  POST="${POST}<ser:entityKey/>"
else
  POST="${POST}<ser:entityKey>${14}</ser:entityKey>"
fi

Now I would change my command line to something more like this.

createRole username password rbpm_url output_file rolename description rolelevel ownerDN approverDN container quorum requestDefDN revokeReqDefDN category correlation_id

That would leave me with a function createRole2, looking something like the following:

### Function: createRole2
# Usage:
# createRole username password rbpm_url output_file rolename description rolelevel ownerDN approverDN container quorum requestDefDN 
#	revokeReqDefDN category correlation_id
# if the $correlation_id is omitted, will call createRoleRequest, otherwise will call createRoleAidRequest
# if the category is omitted with use the value "default"
# Order of parameters is important, it is not possible to use the correlation_id and skip category
# If you have no value for a field other than category and correlationId, use the value of the NULL variable, default is 'null'
createRole2()
{

USAGE="Function Usage:

createRole username password rbpm_url output_file rolename description rolelevel ownerDN approverDN container quorum requestDefDN revokeReqDefDN category correlation_id
 if the correlation_id is omitted, will call createRoleRequest, otherwise will call createRoleAidRequest
 if the category is omitted with use the value "default"
 Order of parameters is important, it is not possible to provide a correlation_id and skip category at the same time
 If you have no value for a field other than category and correlationId, use the value of the NULL variable, default is 'null'
 rbpm_url should be in the format:
  protocol://server:port/servicename
 for example:
  https://rbpm.lab.novell.com:8543/IDMProv";

if "X$_RBPM_SOAP_ROLE_DEBUG" = "Xtrue"
then
  dbgparams=$#
  dbgparam=1
  while [ "$dbgparam" -le "$dbgparams" ]
  do
    echo -n "Parameter " 
    echo -n \$$dbgparam
    echo -n " = "
    eval echo \$$dbgparam
    (( dbgparam   ))
  done
fi

# Initial Parameters check
if | -z "$2" || -z "$3" || -z "$4" || -z "$5" || -z "$6" || -z "$7" || -z "$8" || -z "$9" || -z "$10" || -z "$11" || -z "$12" || -z "$13" || -z "$14"
then
  echo "$USAGE"
  return 1
fi

if -z "$15"
then
  CAT=default
else
  CAT="$8"
fi

if -z "$16"
then
  NOCID=true
else
  NOCID=false
  CID="$9"
fi

if "X$2" = "X-W"
then
  read -sp "Please enter the password for user $1: " SENHA
  echo
else
  SENHA=$2
fi

# Setup for the SOAP call
URL="${3}/role/service"

if "$NOCID" = "true"
then
  ACTION="SOAPAction: 'http://www.novell.com/role/service/createRole'"
  SOAPCALL=createRoleRequest
else
  ACTION="SOAPAction: 'http://www.novell.com/role/service/createRoleAid'"
  SOAPCALL=createRoleAidRequest
fi
CTYPE='Content-Type: text/xml;charset=UTF-8'

# Build SOAP XML envelope and call to be issued
POST="<soapenv:Envelope xmlns:soapenv='http://schemas.xmlsoap.org/soap/envelope/' xmlns:ser='http://www.novell.com/role/service'>\
<soapenv:Header/>\
<soapenv:Body>\
<ser:${SOAPCALL}>\
<ser:role>\
<ser:description>${6}</ser:description>\
<ser:entityKey/>\
<ser:name>${5}</ser:name>\
<ser:roleCategoryKeys>\
<ser:categorykey>\
<ser:categoryKey>${CAT}</ser:categoryKey>\
</ser:categorykey>\
</ser:roleCategoryKeys>\
<ser:roleLevel>${7}</ser:roleLevel>\
<ser:systemRole>false</ser:systemRole>"

if "X$8 = "X$_NULL"
then
  POST="${POST}<ser:owners/>"
else
  POST="${POST}<ser:owners>\
<ser:dnstring>\
<ser:dn>${8}</ser:dn>\
</ser:dnstring>\
</ser:owners>"  
fi

if "X$9 = "X$_NULL"
then
  POST="${POST}<ser:approvers/>"
else
  POST="${POST}<ser:approvers>
<ser:approver>\
<ser:approverDN>${9}</ser:approverDN>\
<ser:sequence>0</ser:sequence>\
</ser:approver>\
</ser:approvers>\
fi

if "X$10 = "X$_NULL"
then
  POST="${POST}<ser:container/>"
else
  POST="${POST}<ser:container>${10}</ser:container>"
fi

if "X$11 = "X$_NULL"
then
  POST="${POST}<ser:quorum/>"
else
  POST="${POST}<ser:quorum>${11}</ser:quorum>"
fi

if "X$12 = "X$_NULL"
then
  POST="${POST}<ser:requestDef/>"
else
  POST="${POST}<ser:requestDef>${12}</ser:requestDef>"
fi

if "X$13 = "X$_NULL"
then
  POST="${POST}<ser:revokeRequestDef/>"
else
  POST="${POST}<ser:revokeRequestDef>${13}</ser:revokeRequestDef>"
fi

if "X$14 = "X$_NULL"
then
  POST="${POST}<ser:entityKey/>"
else
  POST="${POST}<ser:entityKey>${14}</ser:entityKey>"
fi

POST="${POST}</ser:role>"


if "$NOCID" = "false" 
then
  POST="${POST}<ser:correlationId>${CID}</ser:correlationId>"
fi

POST="${POST}</ser:${SOAPCALL}>\
</soapenv:Body>\
</soapenv:Envelope>"

if "X$_RBPM_SOAP_ROLE_DEBUG" = "Xtrue"
then
  echo
  echo POST data:
  echo $POST
  echo
fi

# Issue the request
curl $_CURL_OPTIONS -k -u "$1:$SENHA" -H "$CTYPE" -H "$ACTION" -d "$POST" "$URL" -o "$4"
}

I will send this and my createResource approach back to Fernando for him to consider adding, so you do not have to add it yourself. We shall see what comes of that attempt.

If you find you need additional functionality out of this script either support for additional command line parameters or unsupported SOAP functions, I suggest you contribute it as a comment to Fernando's article or send it to him to add. The more the merrier.