When looking for a bad MARC Record using the rebuild_zebra_sliced.sh, it is
useful to skip the complete MARCXML exporting from Koha and reuse the exported
files for Zebra indexing.
This patch adds a new parameter:
-x | --exclude-export Do not export Biblios from Koha, but use the existing
export-dir
Which depends on the:
-d | --export-dir Where rebuild_zebra.pl will export data
Default: $EXPORTDIR
!---------!
! TEST PLAN !
!---------!
1. Run
"./rebuild_zebra_sliced.sh --length 1000"
to export 1000 MARC Records
and slice them to one big 1000-Record chunk.
2. Realize that you get an imaginary "stack smashing detected"-error crashing
your indexing at some Record you dont know of and can't make out from the
indexing logging.
3. Start looking for the bad Record by running:
"./rebuild_zebra_sliced.sh --exlude-export --chunk-size 10"
To skip Biblios export from Koha which takes ~2h and get straight into
splitting your exported biblios to chunks of 10, and indexing them. You
know which chunk fails so it is much easier to find the issue there.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Default: $CHUNKSSIZE
-d | --export-dir Where rebuild_zebra.pl will export data
Default: $EXPORTDIR
Default: $CHUNKSSIZE
-d | --export-dir Where rebuild_zebra.pl will export data
Default: $EXPORTDIR
+ -x | --exclude-export Do not export Biblios from Koha, but use the existing
+ export-dir
-L | --log-dir Log directory
Default: $LOGDIR
-r | --remove-logs Clean log directory before start
-L | --log-dir Log directory
Default: $LOGDIR
-r | --remove-logs Clean log directory before start
LENGTH=
CHUNKSSIZE=10000
EXPORTDIR=/tmp/rebuild/export
LENGTH=
CHUNKSSIZE=10000
EXPORTDIR=/tmp/rebuild/export
LOGDIR=/tmp/rebuild/logs
RMLOGS=no
NOCONFIRM=no
LOGDIR=/tmp/rebuild/logs
RMLOGS=no
NOCONFIRM=no
+ -x | --exclude-export )
+ EXCLUDEEXPORT=yes
+ ;;
-r | --remove-logs )
RMLOGS=yes
;;
-r | --remove-logs )
RMLOGS=yes
;;
echo "Total number of records to index: $LENGTH"
echo "Initial chunk size: $CHUNKSSIZE"
echo "Export directory: $EXPORTDIR"
echo "Total number of records to index: $LENGTH"
echo "Initial chunk size: $CHUNKSSIZE"
echo "Export directory: $EXPORTDIR"
+echo "Exclude re-exporting: $EXCLUDEEXPORT"
echo "Log directory: $LOGDIR"
echo "Remove logs before start? $RMLOGS"
echo "Type of record: $TYPE"
echo "Log directory: $LOGDIR"
echo "Remove logs before start? $RMLOGS"
echo "Type of record: $TYPE"
-REBUILDZEBRA_CMD="$REBUILDZEBRA $TYPESWITCH -v -x -k -d $EXPORTDIR --offset $OFFSET --length $LENGTH --skip-index"
-echo "\n$REBUILDZEBRA_CMD"
-$REBUILDZEBRA_CMD
+if [ $EXCLUDEEXPORT = "no" ]; then
+ REBUILDZEBRA_CMD="$REBUILDZEBRA $TYPESWITCH -v -x -k -d $EXPORTDIR --offset $OFFSET --length $LENGTH --skip-index"
+ echo "\n$REBUILDZEBRA_CMD"
+ $REBUILDZEBRA_CMD
+fi
EXPORTFILE=
case $TYPE in
EXPORTFILE=
case $TYPE in