Creating your own Document Archive Server on NetWare

0 Likes
I work for a School District, as you may already assume we have a ton of paperwork, some that we have to keep anywhere from 7 years to forever. We literally have a warehouse full of paperwork from the many years our school has been in business.

Our first few attempts at digitizing this paperwork met with complete failure. We allowed the different departments to hire who ever they wanted to scan their documents. What happened was that we got terrible scans and the companies that sold the department on their proprietary document management software would end up going out of business. We finally took the job of picking a document imaging system into the technology department. Unfortunately we did not fair much better, we started with running LaserFiche on NetWare and found that the software was very hard to use and the support was terrible. So the same person that installed the LaserFiche software gave up on them as well and started his own company, of which I no longer remember the name, and got us to buy into his system. The experience was interesting and painful at the same time.

Disgusted with the many document imaging shysters out there we took a step back to figure out what we could do.

Most document imaging systems that that are reliable and have good tech support also cost a whole lot of money but we knew we did not want to go cheap again. So I did some calling around and found a company that just did the conversion of paper to digital documents and he told me something that made a whole lot of sense. He said that I should just scan them to PDF, enter the Metadata into the PDF document itself and just index it with some index engine. To me that made perfect sense and seeing as we had to reconvert and re-import documents from various systems already having a self contained document that was an Industry standard was perfect.

The good news was that Novell came out with this new product called Quickfinder for NetWare and it was exactly what we needed. Comparable indexing engines could cost well over a thousand dollars but Novell’s index engine cost nothing more then the cost of our NetWare servers. Since then Novell has released it on Linux but I have not delved into that yet.

The components we used for our server was iChain, NetWare, Apache and Quickfinder.

I will start with getting the documents to the server.

First I created a NetWare archive server and created a Data volume to house the documents. Now there should be a bit of planning on how you setup your directories. For us I created folders for each department then created folders for each type of document. You may want to create even two more folders under that, one for files burned to a permanent media and ones that have not been burned yet. Once burned to a permanent medium that is non-editable (like a CD RO or DVD RO) it is now a legal copy and you are able to destroy the paper. Just to make clear though, we are a school district and according to our lawyer’s that is the definition of a legal medium as the law is stated now, this may be different for different industries.

So now that the Directories are created you can (if you haven’t already) install Quickfinder. It is easy to do and I have some screen shots for you.







When you install Quickfinder it may ask to install Apache and Tomcat as well if you do not already have that installed.

Once you have installed the Quickfinder server you can start managing it by going to http://<IP Address>/qfsearch/admin . You will by default get the default Index server but you will want to select the global link.



Once in the Global settings you will see this.



What you will want to do when configuring a new index is to make sure the name is the fully qualified domain name of the virtual server you will be using. You can break them up however you will want to make sure the boundary is set to who will have access to the documents. For instance, if you have a bunch of Business documents that only people in the business department will be able to see you will want to make a virtual server that as a name like BusinessDocs.novell.com. This really comes into play later when you restrict direct URL links to the documents.

Once a Virtual server is created you manage it, I will not go into everything for the management, just the things you need to change from defaults to get this to work.



This is the page you get when managing a Virtual Server, here I have created a “New file system index” and pointed the index to a directory that holds documents.

Next you edit an index and get



Here the server path is to the location of the documents but the URL is very important and dependant on how you set up your Apache server, for my Apache server I just created a Apache Virtual server and created additional Doc directories with a URL extension that I matched up here. I will go into configuring the Apache server more later but just remember the secondary Docs area corresponds to this.

When you scroll down the page you have some security options,



You can set this how you want but what I did was just create an empty file called security.txt and gave rights to the file for a group object that users belong to. This setting only, I repeat, only limits search results, if a user knows the complete URL path to a file this does not stop them from accessing the document.

The next step is to go to the Security tab for the Virtual server,



Here you enter the search context for the user IDs, you can also create a realm string but I am unsure what this is for because it does not seem to work the way Novell’s docs say it does, that is authentication does not seem to be passed to Apache for access control on the files, but maybe I am reading the docs wrong on that, anyway, to be safe I change it for each index anyway.

So now the Quickfinder server is done.

So right now you can have people login to Quickfinder, do a search for Docs and assuming the docs fall into a secondary directory in Apache you will get and retrieve results. This is the end if you care nothing for keeping the docs somewhat secure but of course, if you have to save these documents chances are they have sensitive data so you need to go on to the next step. Right now, regardless of what security settings you have on the Quickfinder server, if a person knows the exact URL for a document they can bring it back up completely bypassing Quickfinder security.

The next step is creating and securing the Apache server for these eventualities. So next stop is Apache admin (of course you can do this from directly editing the various config files but I like to do it from the interface then tweak it if need be from the config files). So go to https://:2200/apacheadmin/login.jsp and login.

Once logged in select the Content Management button and you will see



Click the Virtual Hosts link



Select Name based and type the IP address with the port after, so maybe 172.16.3.1:80 and then the name, this name is what corresponds to the Quickfinder Virtual server. So if you are doing this for the BusinessDocs.novell.com server then you have to enter “BusinessDocs.novell.com” in the server name.

Once created you will see



From here you select options for that new virtual server and you get



Select the Additional Document Directories,



The URL Prefix will correspond to the “Corresponding URL Prefix” in each of the Indexes back up top. So in if you defined a index and the URL Prefix you defined in quickfinder was http://BusinessDocs.novell.com/Budegetreport then you will want to enter BudgetReports on the URL Prefix. Next you will enter or navigate to the top most directory of the documents.

Once you enter this you will see it in the list



You will select the Access Control button and get



You will want to select the Auth LDAP Module, use e-Directory rights and enter the Base DN. The rest you can keep as default unless you want Secure LDAP like I opted for. By default the Apache server queries its own LDAP server so the Secure LDAP may not be needed but if it is it should work as configured.

With e-Directory rights configured Apache should look at the user id and see if that user has rights to the files they are trying to view on the server, if they do then you can view them.

As with a lot of technology, you fix one issueand create. If you go into your Quickfinder page now and do a search, then try to access the document you would get another login, which means you now have dual logins?. So to get rid of this we use iChain to front end our Archive server.

The next part goes into configuring iChain for your archives.

The first thing with iChain is to create an Accelerator for your Archive server.



I won’t go into how to configure iChain but if you know how these are the options to check. You want to click the Authentication Options button next..



Once that is set go to the Access Control tab and enter



The important thing here is to enable “Object Level Access Control (OLAC)”

Next stop is the ISO Object in the Tree that holds your iChain configuration.



You will find this on the Protected Resource tab of the ISO object. Once this is set to Secure you go to the OLAC properties and configure it as follows.



Once this all configured you need to create and ACL object for each department and configure it.

Start by creating a new iChain Object,



Then



Then



Now you are ready to edit the ACL, the General tab should look like this



Next Select the Access Control tab



Lastly you need to enter some form fill script.

<!-- Business Archives Login -->
<urlPolicy>
<name>BusinessArchives</name>
<url>businessdocs.novell.com/qfsearch/SearchServlet</url>
<formCriteria>
<input type="text" name="userid" value="" size="20">
</formCriteria>
<actions>
<fill>
<INPUT NAME="userid" value="~cn">
<INPUT NAME="password" value="~password">
<INPUT NAME="login" value="true">
</fill>
<post/>
</actions>
</urlPolicy>



So now you are finally done.

I know what you are thinking, good freaking lord that is a lot of work. In truth it all depends on what you want. We wanted secure access to our archives with Single Sign on as well as make it easy to add users. So once all this is setup all we have to do is copy files into a directory and every night the Index engine runs and updates the index, if someone needs to be added to view the docs we add them to a group and via ZENworks the URL application is pushed to their Desktop and all appropriate rights are given to the user.

Just about every step can be cut out if you do not require the more in depth configuration we do. If you don’t require single sign on and dual sign on is fine then you don’t need the iChain part, if you don’t require a login to access the files then you don’t need the extra Apache access control.

Through all my weeks and months on end to get other peoples Document Management Systems to work I find this to be the best way, with others if there is crash on the server and you need to recover you are talking about days, to weeks to recover. I just migrated this server over to a virtual server and it took less then a day to have it back up and running and if Quickfinder ever does go away there are a lot of other indexing engines out there to use and there is no document conversion involved.

If someone really needs this installed or help feel free to contact me, although I do have a full time job I am always up for working on the side to make little more cash?

You can contact me at blittrell@musd.org or blittrell@comcast.net.

Labels:

How To-Best Practice
Comment List
  • One of the features of QuickFinder that is pretty unique to it, is the ability to honour file system trustees.

    So on a OES (Used to be I would say Netware, but now it can run on SLES with OES as well) server with millions of documents, and hundreds or thousands of users, you can enable Quickfinder, and index all the files. (Including content inside Word, Wordperfect, and PDF files).

    Then you can make the index available to search.

    An anonymous user can search and see basically nothing, or maybe the contents of sys:\login (whatever [Public] trustee has access too).

    Then I login as user Jsmith. I can find files in Jsmith's home directory, files in shared folder he has access to and so on.

    Logout, login in as BWilliams, now I do the exact same search, and I see different results, since I now can only see the contents of the BWilliams home directory, and the shared folders he has file system access to!

    Lovely! This means with one index, you can generate custom search results per user!

    Not many other search engines can do this. The requirement is that the files be stored on an NSS volume on a server in the tree for trustee based searches to work.
Related
Recommended