Make It Live!

From UA Libraries Digital Services Planning and Documentation
(Difference between revisions)
Jump to: navigation, search
m
 
(11 intermediate revisions by 2 users not shown)
Line 1: Line 1:
Here's the workflow for getting Cabaniss content live. (Software here:  [[Cabaniss_Software]])
+
This page documents part of the workflow for "Digitizing the S. D. Cabaniss Papers," an NHPRC-funded grant project.
 +
 
 +
== Preliminary steps on the server ==
 +
 
 +
# make sure that the Windows share is mounted:  type into the ssh window on libcontent (which at the time was libcontent1):  `ls /cifs-mount`  -- if no listing appears, or the window hangs up, you need to mount the share drive.  Otherwise, proceed to getting Cabaniss content live...
 +
# To mount the Windows drive, type this in on the commandline on libcontent (which at the time was libcontent1):  `sudo mount -t cifs -o username=jjcolonnaromano,domain=lib //libfs1.lib.ua-net.ua.edu/share/Digital\ Projects/ /cifs-mount`  and use the password for share for Jeremiah.  If successful, the command in the last step will show you the directories within the Digital Projects folder on Share.
 +
 
 +
 
 +
== Here's the workflow for getting Cabaniss content live ==  
 +
 
 +
(Software here:  [[Cabaniss_Software]];  Diagram of this workflow here:  [[Cabaniss_Papers]])
  
 
1)  starting on the Share drive, make sure all tiffs are in a Scans folder in S:\Digital Projects\Digital_Coll_Complete\u0003_0000252_Cabaniss
 
1)  starting on the Share drive, make sure all tiffs are in a Scans folder in S:\Digital Projects\Digital_Coll_Complete\u0003_0000252_Cabaniss
Line 5: Line 15:
 
2)  use the script S:\Digital Projects\Administrative\scripts\Cabaniss\CabanissCheck.pl to check filenames;  run it till it shows no errors.  [[Image:CabanissCheck.txt]]
 
2)  use the script S:\Digital Projects\Administrative\scripts\Cabaniss\CabanissCheck.pl to check filenames;  run it till it shows no errors.  [[Image:CabanissCheck.txt]]
  
3)  delete all existing jpegs in the upload directory IF they were not deleted after the last upload.
+
3)  open a command-line window in SSH.  Type in: `cd CABANISS/scripts`.  This will change your working directory to the one where the scripts are.  Screenshot of Cabaniss directory structure: [[Image:Cabaniss_directories.PNG]]
  
4)  open a command-line window in SSHType in: `cd CABANISS/scripts`This will change your working directory to the one where the scripts are.
+
4)  run makeCabanissJpegs in CABANISS/scripts directory ([[Image:makeCabanissJpegs.txt]]): type in: `makeCabanissJpegs` This script will ask you which directories you want JPEGs created for (do NOT use for transcripts, or they will overwrite the content images) -- this will create JPEGs for all the tiffs in the directories you've chosen, and will place them all in the CABANISS/jpegs directory on the server.  After creating all the JPEGs it will correct for the multiples created due to thumbnails in the TIFFs.
  
5)  run makeCabanissJpegs in CABANISS/scripts directory:  type in:  `makeCabanissJpegs`  This script will ask you which directories you want JPEGs created for (do NOT use for transcripts, or they will overwrite the content images) -- this will create JPEGs for all the tiffs in the directories you've chosen, and will place them all in the CABANISS/jpegs directory on the serverAfter creating all the JPEGs it will correct for the multiples created due to thumbnails in the TIFFs.
+
5)  Spot check the output. There should be 2 jpegs for every tiff. One ends in _128, and one in _2048. These correspond to the number of maximum pixels to a sideThe one ending in "_128.jpg" is a thumbnail.  The one ending in "_2048.jpg" is a large image used in display.
  
6)  Spot check the output. There should be 2 jpegs for every tiff. One ends in _128, and one in _2048. These correspond to the number of maximum pixels to a side. The one ending in "_128.jpg" is a thumbnailThe one ending in "_2048.jpg" is a large image used in display.
+
6)  run linkContent in CABANISS/scripts directory:  type in: `linkContent` This updates the EAD in the EAD directory and copies the older version to the CABANISS/backups directory. [[Image:LinkContent.txt]]
  
7) run linkContent in CABANISS/scripts directory:  type in: `linkContent` This updates the EAD in the EAD directory and copies the older version to the CABANISS/backups directory. [[Image:LinkContent.txt]]
+
7) check in the output directory for the latest Linking_README file (they are timestamped). Look for "ERROR" and note any warnings. If necessary, make repairs to files and rerunIf the EAD was trashed, the previous copy was placed in the backups directory with a timestamp on it.  You can replace the trashed one with the previous version and rerun if you need to.
  
8) check in the output directory for the latest Linking_README file (they are timestamped)Look for "ERROR" and note any warnings. If necessary, make repairs to files and rerun.  If the EAD was trashed, the previous copy was placed in the backups directory with a timestamp on it.  You can replace the trashed one with the previous version and rerun if you need to.
+
8) run generateMODS in CABANISS/scripts directory:  type in:  `generateMods` This creates minimal mods for each item and places them in the CABANISS/MODS directory -- it also copies them to the deposits directory for archiving with the tiffs. [[Image:GenerateMods.txt]]
  
9)  run generateMODS in CABANISS/scripts directory:  type in:  `generateMods` This creates minimal mods for each item and places them in the CABANISS/MODS directory  -- it also copies them to the deposits directory for archiving with the tiffs. [[Image:GenerateMods.txt]]
+
9)  check in the output directory for the latest MODS_README file (they are timestamped)Look for "ERROR" and note any warnings. If necessary, make repairs to files and rerun. It will overwrite existing MODS only if they have NOT been distributed live.
  
10)  check in the output directory for the latest MODS_README file (they are timestamped).  Look for "ERROR" and note any warnings. If necessary, make repairs to files and rerun.  It will overwrite existing MODS only if they have NOT been distributed live.
+
10)  run makeItLive script in CABANISS/scripts directory ([[Image:makeItLive.txt]]):  type in:  `makeItLive` .  This distributes the jpegs and   
 
+
11)  run makeItLive script in CABANISS/scripts directory:  type in:  `makeItLive` .  This distributes the jpegs and   
+
 
MODS in Acumen.  It will delete the local copies of MODS and jpegs, so you can reuse the same directories over and over.Watch for errors on the command line.
 
MODS in Acumen.  It will delete the local copies of MODS and jpegs, so you can reuse the same directories over and over.Watch for errors on the command line.
  
12run moveCabanissContent script in CABANISS/scripts directory:  type in:  `moveCabanissContent` .  This will check for a collection icon, for collection information (you can upload a new collection xml file if you need to make changes) and it will copy the content to the deposits directory on the storage drive. After checking the copy, it will delete the version on the share drive, including empty scans directories.  Thus, anything left behind did NOT make it to the storage drive for some reason;  don't delete it.  This script also creates a timestamped output file in the CABANISS/outputs directory, and it will put errors there, as well as indicate when it has completed.
+
11move up to the home area, down into the scripts directory there, and run `findMissing` ([[Image:findMissing.txt]]); This script will hunt through Acumen to make sure there is a MODS file for every item, and at least one derivative for each MODS file. Any errors will be found in the output file written to the scripts/output directory. If errors are found, regenerate those MODS and/or JPEGs/MP3s and rerun relocate_all. Then run this script again to ensure all errors have been remedied.
  
13) Wait for indexing to occur.  This may be about 24 hours.  
+
12)  run moveCabanissContent script in CABANISS/scripts directory ([[Image:moveCabanissContent.txt]]):  type in:  `moveCabanissContent` .  This will check for a collection icon, for collection information (you can upload a new collection xml file if you need to make changes) and it will copy the content to the deposits directory on the storage drive.  After checking the copy, it will delete the version on the share drive, including empty scans directories.  Thus, anything left behind did NOT make it to the storage drive for some reason;  don't delete it.  This script also creates a timestamped output file in the CABANISS/outputs directory, and it will put errors there, as well as indicate when it has completed.
 +
 
 +
13) Wait for indexing to occur.  This may be about 24 hours. If you want, you can watch the content being deposited on the storage server in the deposits/Cabaniss directory on the server.  :-)
  
 
14) Check online to locate one of the files you just uploaded.  For example, if one of them was u0003_0000252_2010003, then look for http://acumen.lib.ua.edu/u0003_0000252_2010003  -- if it's there, indexing completed.
 
14) Check online to locate one of the files you just uploaded.  For example, if one of them was u0003_0000252_2010003, then look for http://acumen.lib.ua.edu/u0003_0000252_2010003  -- if it's there, indexing completed.
  
15) Once indexing has completed, run eadLive to copy the EAD to the web directory.  It's important to wait for indexing to be complete so the links in the EAD are valid.
+
15) Once indexing has completed, run eadLive ([[Image:eadLive.txt]])to copy the EAD to the web directory.  It's important to wait for indexing to be complete so the links in the EAD are valid.
  
 
16) Check the output file from the moveCabanissContent script, and also check the directories on the share drive, to see if anything did NOT get moved successfully to the storage drive.  If anything failed, rerun the script;  if it still does not move the content, flag the directory with "_NotInStorage" added to its header, and report it to Jody.
 
16) Check the output file from the moveCabanissContent script, and also check the directories on the share drive, to see if anything did NOT get moved successfully to the storage drive.  If anything failed, rerun the script;  if it still does not move the content, flag the directory with "_NotInStorage" added to its header, and report it to Jody.
Line 37: Line 47:
  
 
18)  exit out of secure shell.  Good work!!
 
18)  exit out of secure shell.  Good work!!
 
 
  Screenshot of Cabaniss directory structure: [[Image:Cabaniss_directories.PNG]]
 
 
Workflow diagram:  [[Cabaniss.png]]
 

Latest revision as of 08:24, 6 August 2013

This page documents part of the workflow for "Digitizing the S. D. Cabaniss Papers," an NHPRC-funded grant project.

[edit] Preliminary steps on the server

  1. make sure that the Windows share is mounted: type into the ssh window on libcontent (which at the time was libcontent1): `ls /cifs-mount` -- if no listing appears, or the window hangs up, you need to mount the share drive. Otherwise, proceed to getting Cabaniss content live...
  2. To mount the Windows drive, type this in on the commandline on libcontent (which at the time was libcontent1): `sudo mount -t cifs -o username=jjcolonnaromano,domain=lib //libfs1.lib.ua-net.ua.edu/share/Digital\ Projects/ /cifs-mount` and use the password for share for Jeremiah. If successful, the command in the last step will show you the directories within the Digital Projects folder on Share.


[edit] Here's the workflow for getting Cabaniss content live

(Software here: Cabaniss_Software; Diagram of this workflow here: Cabaniss_Papers)

1) starting on the Share drive, make sure all tiffs are in a Scans folder in S:\Digital Projects\Digital_Coll_Complete\u0003_0000252_Cabaniss

2) use the script S:\Digital Projects\Administrative\scripts\Cabaniss\CabanissCheck.pl to check filenames; run it till it shows no errors. File:CabanissCheck.txt

3) open a command-line window in SSH. Type in: `cd CABANISS/scripts`. This will change your working directory to the one where the scripts are. Screenshot of Cabaniss directory structure: Cabaniss directories.PNG

4) run makeCabanissJpegs in CABANISS/scripts directory (File:MakeCabanissJpegs.txt): type in: `makeCabanissJpegs` This script will ask you which directories you want JPEGs created for (do NOT use for transcripts, or they will overwrite the content images) -- this will create JPEGs for all the tiffs in the directories you've chosen, and will place them all in the CABANISS/jpegs directory on the server. After creating all the JPEGs it will correct for the multiples created due to thumbnails in the TIFFs.

5) Spot check the output. There should be 2 jpegs for every tiff. One ends in _128, and one in _2048. These correspond to the number of maximum pixels to a side. The one ending in "_128.jpg" is a thumbnail. The one ending in "_2048.jpg" is a large image used in display.

6) run linkContent in CABANISS/scripts directory: type in: `linkContent` . This updates the EAD in the EAD directory and copies the older version to the CABANISS/backups directory. File:LinkContent.txt

7) check in the output directory for the latest Linking_README file (they are timestamped). Look for "ERROR" and note any warnings. If necessary, make repairs to files and rerun. If the EAD was trashed, the previous copy was placed in the backups directory with a timestamp on it. You can replace the trashed one with the previous version and rerun if you need to.

8) run generateMODS in CABANISS/scripts directory: type in: `generateMods` . This creates minimal mods for each item and places them in the CABANISS/MODS directory -- it also copies them to the deposits directory for archiving with the tiffs. File:GenerateMods.txt

9) check in the output directory for the latest MODS_README file (they are timestamped). Look for "ERROR" and note any warnings. If necessary, make repairs to files and rerun. It will overwrite existing MODS only if they have NOT been distributed live.

10) run makeItLive script in CABANISS/scripts directory (File:MakeItLive.txt): type in: `makeItLive` . This distributes the jpegs and MODS in Acumen. It will delete the local copies of MODS and jpegs, so you can reuse the same directories over and over.Watch for errors on the command line.

11) move up to the home area, down into the scripts directory there, and run `findMissing` (File:FindMissing.txt); This script will hunt through Acumen to make sure there is a MODS file for every item, and at least one derivative for each MODS file. Any errors will be found in the output file written to the scripts/output directory. If errors are found, regenerate those MODS and/or JPEGs/MP3s and rerun relocate_all. Then run this script again to ensure all errors have been remedied.

12) run moveCabanissContent script in CABANISS/scripts directory (File:MoveCabanissContent.txt): type in: `moveCabanissContent` . This will check for a collection icon, for collection information (you can upload a new collection xml file if you need to make changes) and it will copy the content to the deposits directory on the storage drive. After checking the copy, it will delete the version on the share drive, including empty scans directories. Thus, anything left behind did NOT make it to the storage drive for some reason; don't delete it. This script also creates a timestamped output file in the CABANISS/outputs directory, and it will put errors there, as well as indicate when it has completed.

13) Wait for indexing to occur. This may be about 24 hours. If you want, you can watch the content being deposited on the storage server in the deposits/Cabaniss directory on the server.  :-)

14) Check online to locate one of the files you just uploaded. For example, if one of them was u0003_0000252_2010003, then look for http://acumen.lib.ua.edu/u0003_0000252_2010003 -- if it's there, indexing completed.

15) Once indexing has completed, run eadLive (File:EadLive.txt)to copy the EAD to the web directory. It's important to wait for indexing to be complete so the links in the EAD are valid.

16) Check the output file from the moveCabanissContent script, and also check the directories on the share drive, to see if anything did NOT get moved successfully to the storage drive. If anything failed, rerun the script; if it still does not move the content, flag the directory with "_NotInStorage" added to its header, and report it to Jody.

17) Check the website and make sure everything is hunky dory.  :-)

18) exit out of secure shell. Good work!!

Personal tools