Highlighted
Super Contributor.
Super Contributor.
331 views

Identity Application PostgreSQL database reinitialization

Jump to solution

Hi,

 

I have a test system upgraded to IDM 4.8.1 with IA 4.8.1 all on one SLES 15 SP1, freshly installed. When doing the post-upgrade task (upgrading PostgreSQL to 12.2) I noticed locale of PostgreSQL database is set to "sl_SI.UTF-8", but upgrade script had "en_US.UTF-8" locale hardcoded. No problem there I changed script's hardcoded locale to "sl_SI.UTF-8" and the upgrade succeeded everything works and I have a working system.

Now for bulletproofing future upgrades, I want to change my IA's PostgreSQL's locale from "sl_SI.UTF-8" to "en_US.UTF-8". 

So what I did is, I used pg_dumpall to "export" and "import" the database. As described here https://www.postgresql.org/docs/12/app-pg-dumpall.html

So I exported the database. Than deleted "/opt/netiq/idm/postgres/data" directory. Then I just did steps 4/j. - 4/l. (https://www.netiq.com/documentation/identity-manager-47/setup_linux/data/migrating-identity-applications.html). I also tried adding rights for the workflow database (in one of my countless tries). In this step, I created a database with "en_US.UTF-8" locale then I imported database with "sl_SI.UTF-8" locale into it (pg_dumpall in the first link) (not the brightest idea I am realizing after writing this, but I guess it was worth a try).

I restarted the whole virtual machine but after login in Identity Application nothing loads up, therefore, due to lack of documentation and forum posts (at least I wasn't able to find any), I am abandoning this "solution". If anyone has any experience with it or sees an important step missing please let me know. This idea is supposed to follow a recommendation from this post https://stackoverflow.com/a/20279796

The new solution is just creating a new PostgreSQL database. So the actual question is:

TL;DR: How do I recreate the PostgreSQL database for Identity Application from scratch?

So to explain a bit more, the idea is what to do if you, say accidentally delete the PostgreSQL database. Is there a documentation of manually installing and configuring PostgreSQL for Identity Application. 

The last thing I want to add I don't have much practical experience with databases so if I did any stupid assumptions please let me know.

 

Thanks in advance

Kind regards

Žan

1 Solution

Accepted Solutions
Highlighted
Outstanding Contributor.
Outstanding Contributor.

Hi Zan,

If you go the the file : 

/opt/netiq/idm/apps/tomcat/conf/ism-configuration.properties,

you have this parameter: 

com.netiq.idm.create-db-on-startup = false

I think you should initialize your postgresql DB with : initdb, then set the parameter to TRUE and restart tomcat.

Hope this will help.

 

Sylvain

View solution in original post

7 Replies
Highlighted
Outstanding Contributor.
Outstanding Contributor.

Hi Zan,

If you go the the file : 

/opt/netiq/idm/apps/tomcat/conf/ism-configuration.properties,

you have this parameter: 

com.netiq.idm.create-db-on-startup = false

I think you should initialize your postgresql DB with : initdb, then set the parameter to TRUE and restart tomcat.

Hope this will help.

 

Sylvain

View solution in original post

Highlighted
Super Contributor.
Super Contributor.

Hi sma2006,

first thanks for your response. Unfortunately, your solution did not work for me ☹️

What I did is:
From everything working (PostgreSQL locale=sl_SI.UTF-8) I first stoped PostgreSQL:
/opt/netiq/idm/postgres/bin/pg_ctl -D /opt/netiq/idm/postgres/data -l logfile stop
then I removed everything from PostgreSQL directory instance:
rm -r /opt/netiq/idm/postgres/data/*
than I initialized the database:
LANG=en_US.UTF-8 /opt/netiq/idm/postgres/bin/initdb -D /opt/netiq/idm/postgres/data
I started the PostgreSQL instance (as the message after initialization says)
/opt/netiq/idm/postgres/bin/pg_ctl -D /opt/netiq/idm/postgres/data -l logfile start
I also checked com.netiq.idm.create-db-on-startup parameter in file /opt/netiq/idm/apps/tomcat/conf/ism-configuration.properties but it was already set on true...
and restarted the tomcat
systemctl restart netiq-tomcat.service

 

Few notes:

com.netiq.idm.create-db-on-startup attribute is from start (everything working) set on true.

I tried reinitializing without deleting /opt/netiq/idm/postgres/data/* and it failed.

Also tried deleting without stoping /opt/netiq/idm/postgres/data, but had an error when initializing new database.

Command systemctl restart netiq-tomcat.service actually stops IA...

Also tried with restarting the whole server but did not work either...

What am I missing?

Highlighted
Knowledge Partner
Knowledge Partner

There is a Java command using a tool called Liquibase that IDM and IG use that read XML files that define the DB Config.  The installer uses it.  The command used to be in the docs at one point.  I do not have it handy.  Rerun the 4.8 installer and have it create the DB then is the easiest way to get it going.

0 Likes
Highlighted
Super Contributor.
Super Contributor.

Hi, you probably meant this https://www.netiq.com/documentation/identity-manager-47/setup_linux/data/configuring-identity-applications.html#b1bc935v

But I cannot find NetIQ-Custom-Install.log probably because in 4.8 that file is deleted for safety I recall reading somewhere about something like that.

Of course, that did not stop me I tried to modify example stated for my needs but did not work 😞

 

This is the result of my command:

Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
at liquibase.logging.core.Slf4JLoggerFactory.getLog(Slf4JLoggerFactory.java:9)
at liquibase.logging.LogService.getLog(LogService.java:39)
at liquibase.integration.commandline.Main.<clinit>(Main.java:61)
Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 3 more

 

I do not know what you meant by using 4.8 installer?

When I ran it just said I can install stuff I don't have like IGA, but no option to just install PostgreSQL for Identity application as it recognizes it and won't do anything else with it (except report it as installed).

 

Highlighted
Knowledge Partner
Knowledge Partner

So yes, the link is what I meant, but the db.out or log file is not usually persisted.  (*They do remove the password from it which helps).

Either you need another classpath statement to include the logging JAR or in your path.

As for the installer idea...  There are multiple parts...  You do not need to reinstall Postgress, the binaries to run the DB itself.

You need to run the ID Apps installer to make it call the command to talk to the previously installed DB and create the needed table and views.

 

0 Likes
Highlighted
Super Contributor.
Super Contributor.

I am slowly giving up 😅

/opt/netiq/common/jre/bin/java -Xms256m -Xmx256m -Dwar.context.name=IDMProv -Ddriver.dn="cn=User Application Driver,cn=driverset1,ou=system,o=IDV" -Duser.container="ou=data,o=IDV" -jar /opt/netiq/idm/apps/UserApplication/liquibase.jar --databaseClass=liquibase.database.core.PostgresDatabase --driver=org.postgresql.Driver --classpath=/opt/netiq/idm/postgres/postgresql-9.4.1212.jar opt/netiq/idm/apps/tomcat/webapps/IDMProv.war --changeLogFile=/opt/netiq/idm/apps/tomcat/webapps/workflow/WEB-INF/classes/DatabaseChangeLog.xml --url="jdbc:postgresql://localhost:5432/idmuserappdb" --contexts="prov,newdb" --logLevel=info --logFile=/opt/netiq/idm/apps/UserApplication/db.out --username=postgres --password=******** updateSQL

/opt/netiq/common/jre/bin/java -Xms256m -Xmx256m -Dwar.context.name=IDMProv -Ddriver.dn="cn=User Application Driver,cn=driverset1,ou=system,o=IDV" -Duser.container="ou=data,o=IDV" -jar /opt/netiq/idm/apps/UserApplication/liquibase.jar --databaseClass=liquibase.database.core.PostgresDatabase --driver=org.postgresql.Driver --classpath=/opt/netiq/idm/postgres/postgresql-9.4.1212.jar /opt/netiq/idm/apps/tomcat/webapps/IDMProv.war /opt/netiq/idm/apps/UserApplication/liquibase/lib/logback-classic-1.2.3.jar /opt/netiq/idm/apps/UserApplication/liquibase/lib/logback-core-1.2.3.jar /opt/netiq/idm/apps/UserApplication/liquibase/lib/slf4j-api-1.7.26.jar /opt/netiq/idm/apps/UserApplication/liquibase/lib/slf4j-simple-1.7.26.jar --changeLogFile=/opt/netiq/idm/apps/tomcat/webapps/workflow/WEB-INF/classes/DatabaseChangeLog.xml --url="jdbc:postgresql://localhost:5432/idmuserappdb" --contexts="prov,newdb" --logLevel=info --logFile=/opt/netiq/idm/apps/UserApplication/db.out --username=postgres --password=******* updateSQL

Only a few of my tries, I tried every command with and without SQL at the end (in bold in the second command so you know what I am talking about). Also, I tried putting classpath in "" brackets and separating them with ; but that did not work either.

I am eager to find a solution but I cannot waste too much time on it.

 

Regarding  installer idea: I did several searches for certain commands and found out I need to run this:

/tmpCD/user_application # ./install.sh
scripts/ua_configure.sh: line 966: : No such file or directory
../common/scripts/config_utils.sh: line 35: ${LOG_FILE_NAME}: ambiguous redirect
../common/scripts/config_utils.sh: line 35: ${LOG_FILE_NAME}: ambiguous redirect
../common/scripts/config_utils.sh: line 35: ${LOG_FILE_NAME}: ambiguous redirect
../common/scripts/config_utils.sh: line 35: ${LOG_FILE_NAME}: ambiguous redirect
../common/scripts/config_utils.sh: line 35: ${LOG_FILE_NAME}: ambiguous redirect
../common/scripts/config_utils.sh: line 35: ${LOG_FILE_NAME}: ambiguous redirect
../common/scripts/config_utils.sh: line 35: ${LOG_FILE_NAME}: ambiguous redirect
../common/scripts/config_utils.sh: line 35: ${LOG_FILE_NAME}: ambiguous redirect
../common/scripts/config_utils.sh: line 35: ${LOG_FILE_NAME}: ambiguous redirect
../common/scripts/config_utils.sh: line 35: ${LOG_FILE_NAME}: ambiguous redirect
../common/scripts/config_utils.sh: line 35: ${LOG_FILE_NAME}: ambiguous redirect
../common/scripts/config_utils.sh: line 35: ${LOG_FILE_NAME}: ambiguous redirect
../common/scripts/config_utils.sh: line 35: ${LOG_FILE_NAME}: ambiguous redirect
../common/scripts/config_utils.sh: line 35: ${LOG_FILE_NAME}: ambiguous redirect
../common/scripts/config_utils.sh: line 35: ${LOG_FILE_NAME}: ambiguous redirect
../common/scripts/config_utils.sh: line 35: ${LOG_FILE_NAME}: ambiguous redirect
../common/scripts/config_utils.sh: line 35: ${LOG_FILE_NAME}: ambiguous redirect
../common/scripts/config_utils.sh: line 35: ${LOG_FILE_NAME}: ambiguous redirect
scripts/ua_configure.sh: line 1015: : No such file or directory
Install supported from wrapper level only. Try running the install.sh @ /tmpCD/user_application/../

 

Did not work for me upon further investigation of install.sh I found out I could run something like this:

./install.sh ${PARAM_STR} -prod $PROD_NAME -wci -log ${LOG_FILE_NAME}

but I don't know what should I replace ${PARAM_STR} and $PROD_NAME with...

 

Highlighted
Super Contributor.
Super Contributor.

Ok, so sma2006's answer actually helped and it is the solution to my main question. Now let me extrapolate a bit more.

 

What worked for me is:

Create a database on startup worked when I dropped idmuserappdb and igaworkflowdb databases. Then I had to create them when I did that they didn't have any content, and this is what I think "create on startup" did. This might not be result of com.netiq.idm.create-db-on-startup, this might be another automatic feature, but I assume it is a feature of com.netiq.idm.create-db-on-startup.

Now, this did not solve my problem. Therefore I am going to make a big heading so everybody looking for the answer to my exact problem finds it, but I had to accept sma2006  answer as a solution because it was.

 

MY SOLUTION:

So the solution is done with two users ROOT and POSTGRES. I think my implementation makes sure everything is ok with the IA databases as it rebuilds them with the new locale. Now me not knowing databases that well I am searching for a quick and easy solution on how to check that database cluster is really ok and healthy, that might be a bit off this forum's topic but I can try?

 

ROOT:
systemctl stop netiq-tomcat.service      #first we stop IA
---------------------------------------------------------------------------------------------------------
POSTGRES:

psql                                                              #we drop databases
\l
drop database idmuserappdb;
drop database igaworkflowdb;

/opt/netiq/idm/postgres/bin/pg_ctl -D /opt/netiq/idm/postgres/data -l logfile stop     #we stop postgres database
cat /opt/netiq/idm/postgres/data/postgresql.conf | grep locale                             #we look up current locale
grep -rnwi /opt/netiq/idm/postgres/data/ -e 'sl_SI.UTF-8'                           #we find current locale occurences
find /opt/netiq/idm/postgres/data/ -type f -exec sed -i 's/sl_SI.UTF-8/en_US.UTF-8/g' {} + #we replace current locale 
grep -rnwi /opt/netiq/idm/postgres/data/ -e 'en_US.UTF-8'                    #wecheck replaced locale occurences
/opt/netiq/idm/postgres/bin/pg_ctl -D /opt/netiq/idm/postgres/data -l logfile start   #we start postgres 

psql                                                    #we create databases and change owners (source/idea from my first post)
\l
createdb idmuserappdb
createdb igaworkflowdb
ALTER DATABASE idmuserappdb OWNER TO idmadmin;
ALTER DATABASE igaworkflowdb OWNER TO idmadmin;
---------------------------------------------------------------------------------------------------------
ROOT:
systemctl start netiq-tomcat.service     #we start IA

 

The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.