Question regarding Elasticsearch re-indexing for upgrade to Octane 16.2.100

Hi.

I am in the process of upgrading Octane 16.1.100 to 16.2.100. This also means upgrading Elasticsearch from 7.16.2 to 8.6.0. In the upgrade guide it is mentioned that an Elasticsearch re-indexing needs to be done first.

My questions are

  • Does the re-indexing need to be run in Octane 16.1.100 or can I do it after upgrading to 16.2.100
  • Does the re-indexing need to be run while still using Elasticsearch 7.16.2 or can I upgrade it first to 8.6.0
  • Also the upgrade guide for 16.1.100 already mentions the re-indexing although the official support matrix for Octane 16.1.100 only mentions Elasticsearch 7.9.2  and 7.16.2 as supported versions. Is this a gap in the documentation?
  • The upgrade guide for Elasticsearch 8.6 mentions that Elasticsearch 7.17 needs to be in place first. Can this be omitted because of the re-indexing?

Answers to those question are much appreciated.

Best regards
Silvio

  • Verified Answer

    One comment prior to replying to the specific questions:

    Reindex in Elastic is a service that moves data from one index to another one. In Octane we have a process (post-upgrade job) in which, per each existing index: a new index is created, the data is reindexed from the existing index to the new one and the existing one is removed. This is required as a preparation stage before upgrading elastic to a new major version since elastic supports indexes created in the major version of the elastic server you are working with or one version back. Also please note, that Octane uses such reindex cycles to add index enhancements that require reindex.
    Also note that reindex can be a long duration task depending on index sizes. For example, in the Octane instance used by the Octane dev group, we have an index that takes 30+ hours to reindex. The good news is that in Octane 16.1.100, reindex was modified to be a near-zero downtime process. I.e., there are small periods during a reindex in which the index’s features are in downtime. For example, in those 30+ hours of reindex mentioned above, the related features were in a <10-minute downtime.

     

    • Does the re-indexing need to be run in Octane 16.1.100 or can I do it after upgrading to 16.2.100

    It could be done in 16.2.100, but since 16.1.200 there is an enforcement check that reindex has already been done. So non-reindexed spaces will not be accessible by users until reindex is completed on them. This means that users will not benefit from the near-zero-downtime nature of the Octane reindex process, and they will be blocked out of the space for the full duration of reindex. (we have quite a few customers where the whole process for their large spaces is <2 hours, but, as mentioned above, it could take much more, depending on the index sizes)

    • Does the re-indexing need to be run while still using Elasticsearch 7.16.2 or can I upgrade it first to 8.6.0

    Yes, it needs to run while elastic is in version 7. Note that if you happen to have indexes from version 6, you will not be able to upgrade elastic to version 8.

    • Also the upgrade guide for 16.1.100 already mentions the re-indexing although the official support matrix for Octane 16.1.100 only mentions Elasticsearch 7.9.2  and 7.16.2 as supported versions. Is this a gap in the documentation?

    As mentioned above, reindex is a preparation stage before upgrading elastic. So, in preparation for upgrading to elastic 8 we execute reindex (while still using elastic 7). And not a documentation gap Blush.