AEM: SegmentNotFoundException Segment not found

A SegmentNotFoundException in the error log means a segment is not present any more although we are still trying to access it.

To resolve this, please follow the below steps:

Download oak-run jar file from here http://repo1.maven.org/maven2/org/apache/jackrabbit/oak-run/
* AEM6.0 - If using Oak 1.0.12 or later then use Oak 1.1.8 - oak-run-1.1.8.jar. If the Oak version is 1.0.11 or earlier then use oak-run 1.1.6.
Note: We are using a 1.1.x branch version of oak-run because 1.0.x branch doesn't have the "check" command implemented.

1) Stop AEM

2) Run this command:
java -jar oak-run-*.jar check -d1 --bin=-1 -p crx-quickstart/repository/segmentstore/
It which will search backwards through the revisions until it finds a consistent one, Example:
14:00:30.783 [main] INFO o.a.j.o.p.s.f.t.ConsistencyChecker - Found latest good revision afdb922d-ba53-4a1b-aa1b-1cb044b535cf:234880

3)Revert the repository to this revision by editing ./crx-quickstart/repository/segmentstore/journal.log. Delete all lines after the line containing the latest good revision.
If you would like to find out what date and time you are reverting the repository to then run this command in the segmentstore folder (replace afdb922d-ba53-4a1b-aa1b-1cb044b535cf with the latest good revision in your journal.log):
find . -type f -name "data*.tar" -exec sh -c "tar -tvf {} |grep afdb922d-ba53-4a1b-aa1b-1cb044b535cf" \; -print
The output would show you an approximate date and time of that revision.

4)Remove all ./crx-quickstart/repository/segmentstore/*.bak files.

5)If using AEM6.0 then download the oak-run version matching what is installed in AEM for the remaining steps. Download it from here http://repo1.maven.org/maven2/org/apache/jackrabbit/oak-run/

6)Run checkpoint clean-up to remove orphaned checkpoints:
java -jar oak-run-*.jar checkpoints ./crx-quickstart/repository/segmentstore rm-unreferenced

7)Finally compact the repository:
java -jar oak-run-*.jar compact ./crx-quickstart/repository/segmentstore/


After the Above steps are completed, your system should now boot up properly.

Utility for taking backup of AEM contents

This utility is in the form of a unix shell script. The script will take the backup of AEM contents present in production author instance, create a zip file of it, download it and keep it in a particular location as defined in the script. Once the process is completed, an email will be sent to users to let them know the status of the job, whether the process was a success or resulted in an error. To automate this entire process, a cronjob can be set up which needs to be scheduled.

The technical document and the script are attached in a zip file. Check the downloads section for the zip file.