Welcome Serena Central users! CLICK HERE
The migration of the Serena Central community is currently underway. Be sure to read THIS MESSAGE to get your new login set up to access your account.
chasb73 Absent Member.
Absent Member.
3090 views

Unable to delete Bundles

Hi

Whenever we want to delete a bundle or policy our ZCM just goes into a hung state.

Anyone else have this issue?

10.3 is now installed
W2K3 server
MSSQL external DB

Cheers
Labels (2)
0 Likes
14 Replies
fishoil Absent Member.
Absent Member.

Re: Unable to delete Bundles

any error messages?
0 Likes
chasb73 Absent Member.
Absent Member.

Re: Unable to delete Bundles

fishoil;1990413 wrote:
any error messages?


Not that i can see. Browser just hangs. I eventually have to restart the services on the server... which results in nothing changing. The bundle in question remains, and users can log in through zenworks again.

Given up trying to delete them now!

Could post log file next time i try it? Not sure which log tho...
0 Likes
patrick_reeder Absent Member.
Absent Member.

Re: Unable to delete Bundles

chasb73;1990658 wrote:
Not that i can see. Browser just hangs. I eventually have to restart the services on the server... which results in nothing changing. The bundle in question remains, and users can log in through zenworks again.

Given up trying to delete them now!

Could post log file next time i try it? Not sure which log tho...


How does your database look? You running weekly maintenance? Also, does it matter on the size of the bundle?

DEV - ZCM 10.3.4/MS-SQL Database (2 Primaries & 2 Sats running 64-bit SLES 10.2 w/ ENGL Imaging Toolkit 7) PROD - ZCM 10.3.4/MS-SQL Database (5 Primaries & 11 Sats running 64-bit SLES 10.2 w/ ENGL Imaging Toolkit 7 ~7500 Managed Devices)
0 Likes
chasb73 Absent Member.
Absent Member.

Re: Unable to delete Bundles

Running weekly maintenance, yeh.

DB = 45GB ish in size

Bundle size doesnt matter, have some policies i want to remove too, they hang it just the same!

Frustrating
0 Likes
patrick_reeder Absent Member.
Absent Member.

Re: Unable to delete Bundles

chasb73;1990705 wrote:
Running weekly maintenance, yeh.

DB = 45GB ish in size

Bundle size doesnt matter, have some policies i want to remove too, they hang it just the same!

Frustrating


How large is your ZCM deployment (number of managed devices, etc.)?

DEV - ZCM 10.3.4/MS-SQL Database (2 Primaries & 2 Sats running 64-bit SLES 10.2 w/ ENGL Imaging Toolkit 7) PROD - ZCM 10.3.4/MS-SQL Database (5 Primaries & 11 Sats running 64-bit SLES 10.2 w/ ENGL Imaging Toolkit 7 ~7500 Managed Devices)
0 Likes
patrick_reeder Absent Member.
Absent Member.

Re: Unable to delete Bundles

Reason I ask, is you might be having some database issues. If you have workstations that take forever to register/unregister with the zone, or slow responsiveness in the ZCC, that tends to point to heavy database fragmentation (or large size).

Check out the following: The zStatusEvent and zObjectInfo tables may grow extremely large over time

We were having very sluggish performance ZCC, logins, refreshes, etc. That's when I noticed that the primary servers were floaded with milions of status files that they were still trying to process to upload to the database. We ended up having to write a bash script to clean up the status messages on the primaries, then run queries against the database to cleanup all the old events from the tables.

After the cleanup, our database went from over 130 GBs to just over 9 GBs. Dramatic improvement where seen after the database got under control.

DEV - ZCM 10.3.4/MS-SQL Database (2 Primaries & 2 Sats running 64-bit SLES 10.2 w/ ENGL Imaging Toolkit 7) PROD - ZCM 10.3.4/MS-SQL Database (5 Primaries & 11 Sats running 64-bit SLES 10.2 w/ ENGL Imaging Toolkit 7 ~7500 Managed Devices)
0 Likes
chasb73 Absent Member.
Absent Member.

Re: Unable to delete Bundles

That sounds like it would be worth looking into, although we are upgraded to 10.3.

We've got 2000 machines, and about 250 bundles from memory in the zone, so not sure how large the DB should be!!
0 Likes
patrick_reeder Absent Member.
Absent Member.

Re: Unable to delete Bundles

chasb73;1990774 wrote:
That sounds like it would be worth looking into, although we are upgraded to 10.3.

We've got 2000 machines, and about 250 bundles from memory in the zone, so not sure how large the DB should be!!


We have about 5400 managed devices, 740 bundles, about 1K inventory only devices and our database is about 9 GB just for comparison.

Looks like your zSatusEvent table is pretty large. I think we had about 12 million or more records in there before the cleanup.

Not that this is your issue, just sounds an awful like what we experienced when our database was clogged up.

DEV - ZCM 10.3.4/MS-SQL Database (2 Primaries & 2 Sats running 64-bit SLES 10.2 w/ ENGL Imaging Toolkit 7) PROD - ZCM 10.3.4/MS-SQL Database (5 Primaries & 11 Sats running 64-bit SLES 10.2 w/ ENGL Imaging Toolkit 7 ~7500 Managed Devices)
0 Likes
chasb73 Absent Member.
Absent Member.

Re: Unable to delete Bundles

patrick_reeder;1990768 wrote:
Reason I ask, is you might be having some database issues. If you have workstations that take forever to register/unregister with the zone, or slow responsiveness in the ZCC, that tends to point to heavy database fragmentation (or large size).

Check out the following: The zStatusEvent and zObjectInfo tables may grow extremely large over time

We were having very sluggish performance ZCC, logins, refreshes, etc. That's when I noticed that the primary servers were floaded with milions of status files that they were still trying to process to upload to the database. We ended up having to write a bash script to clean up the status messages on the primaries, then run queries against the database to cleanup all the old events from the tables.

After the cleanup, our database went from over 130 GBs to just over 9 GBs. Dramatic improvement where seen after the database got under control.


When i run:

select count(*) from zstatusevent;
select count(*) from zObjectInfo where id not in (select objinfoid from zStatusEvent);

it returns:

6628065
0

So it looks like that's not the issue...

We backup + shrink DB daily, and rebuild indexes weekly
0 Likes
patrick_reeder Absent Member.
Absent Member.

Re: Unable to delete Bundles

Also, check to see how many files are in the following folder on each primary in your zone:

/var/opt/novell/zenworks/collection/status

On each of our primaries we had over a million files waiting to be processed, all the way back since last year. The servers were never able to finish processing the status messages so they got caught in an endless loop.

DEV - ZCM 10.3.4/MS-SQL Database (2 Primaries & 2 Sats running 64-bit SLES 10.2 w/ ENGL Imaging Toolkit 7) PROD - ZCM 10.3.4/MS-SQL Database (5 Primaries & 11 Sats running 64-bit SLES 10.2 w/ ENGL Imaging Toolkit 7 ~7500 Managed Devices)
0 Likes
chasb73 Absent Member.
Absent Member.

Re: Unable to delete Bundles

patrick_reeder;1990818 wrote:
Also, check to see how many files are in the following folder on each primary in your zone:

/var/opt/novell/zenworks/collection/status

On each of our primaries we had over a million files waiting to be processed, all the way back since last year. The servers were never able to finish processing the status messages so they got caught in an endless loop.


The collection folder does indeed have a lot of files in it. What did you do with them? I can't believe its safe to just delete them!?

I'm running:

delete from zObjectInfo where id not in (select objinfoid from zStatusEvent);

to clear all the old data up too, but this is meant to be fixed in 10.2.1 and newer so i'm a bit concerned i'm doing more harm than good!! Backup taken, can always revertt to that!!
0 Likes
patrick_reeder Absent Member.
Absent Member.

Re: Unable to delete Bundles

chasb73;1991116 wrote:
The collection folder does indeed have a lot of files in it. What did you do with them? I can't believe its safe to just delete them!?

I'm running:

delete from zObjectInfo where id not in (select objinfoid from zStatusEvent);

to clear all the old data up too, but this is meant to be fixed in 10.2.1 and newer so i'm a bit concerned i'm doing more harm than good!! Backup taken, can always revertt to that!!


We had an SR in when we started having extremly slow logins and poor performance in the ZCC. The engineer told us to remove those status files from the server first before we ran the cleanup queries on the database. That allowed them to breathe a little. What you are cleaning up is only the status events for bundle deployments (when you have a bundle's summary page open, the status section on that page). The great thing about cleaning up the entries, is they will repopulate once workstations refresh/check-in with the zone. If you are hesitant to do any cleanup, I'd go ahead and call in an SR. We were not charged an incident since it was a bug inherent in the later code. I think our issue was we went from 10.2.0 to 10.2.2. When we ran queries on the size of those tables in question, it looked like the issue was there long before we decided to move to 10.2.2.

You are definitely better off than we were. Our database was 130GB and we had about 12 million entries in the zstatusevent table. Be sure you have plenty of space on the partition that contains your database. I'd say two to three times the size of your database. When I ran the cleanup query we actually ran out of space. Luckily we have ours on a SAN, so we were able to allocate more space then extend the partition. Definitely something you want to run on the weekend.

DEV - ZCM 10.3.4/MS-SQL Database (2 Primaries & 2 Sats running 64-bit SLES 10.2 w/ ENGL Imaging Toolkit 7) PROD - ZCM 10.3.4/MS-SQL Database (5 Primaries & 11 Sats running 64-bit SLES 10.2 w/ ENGL Imaging Toolkit 7 ~7500 Managed Devices)
0 Likes
Highlighted
chasb73 Absent Member.
Absent Member.

Re: Unable to delete Bundles

patrick_reeder;1991724 wrote:
We had an SR in when we started having extremly slow logins and poor performance in the ZCC. The engineer told us to remove those status files from the server first before we ran the cleanup queries on the database. That allowed them to breathe a little. What you are cleaning up is only the status events for bundle deployments (when you have a bundle's summary page open, the status section on that page). The great thing about cleaning up the entries, is they will repopulate once workstations refresh/check-in with the zone. If you are hesitant to do any cleanup, I'd go ahead and call in an SR. We were not charged an incident since it was a bug inherent in the later code. I think our issue was we went from 10.2.0 to 10.2.2. When we ran queries on the size of those tables in question, it looked like the issue was there long before we decided to move to 10.2.2.

You are definitely better off than we were. Our database was 130GB and we had about 12 million entries in the zstatusevent table. Be sure you have plenty of space on the partition that contains your database. I'd say two to three times the size of your database. When I ran the cleanup query we actually ran out of space. Luckily we have ours on a SAN, so we were able to allocate more space then extend the partition. Definitely something you want to run on the weekend.


Bloody Windows Servers, just hanging when trying to get into that folder:

%ZENWORKSHOME%\Novell\ZENworks\work\collection\status

Assume thats the one on a Windows box, have cleared up the DB overnight, but will try to clean this folder up and rerun the queries against the DB

I will streamline this bloody thing!!!!!!

Cheers for your help so far!
0 Likes
patrick_reeder Absent Member.
Absent Member.

Re: Unable to delete Bundles

That's funny....we use Linux satellites, and it would crash my WinSCP sessiong when I tried to access that folder. lol! Good times.

What we did was run an ls command parsed using grep so the output was passed to a test file. From there, we created a bash script to remove the files. There were just too many in there for the GUI/command-interface to display. Hopefully that shows some improvements for ya. Be sure to watch your ZENworks primary servers' logs to make sure they are processing any current status files correctly without issue.

They show up in the loader-messages.log as:

[DEBUG] [6/16/10 6:57:39 AM] [] [Loader.Status Storer Module] [] [Processing Status_2b572ec266520f42b46b7e1a314ce073_1276689394355.xml] [] []

If you look in that log right now, that's probably all you see. At least that's what I saw on our primaries...just a constant stream of files processing...non-stop.

Once you clean-up that folder, you should only see a few dozen in intervals after that. Good luck and let me know if you have any questions.

DEV - ZCM 10.3.4/MS-SQL Database (2 Primaries & 2 Sats running 64-bit SLES 10.2 w/ ENGL Imaging Toolkit 7) PROD - ZCM 10.3.4/MS-SQL Database (5 Primaries & 11 Sats running 64-bit SLES 10.2 w/ ENGL Imaging Toolkit 7 ~7500 Managed Devices)
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.