For Creating Derivatives - Revision history

Scpstu32: /* The scripts currently used for creating derivatives */

Scpstu32 — Wed, 13 Oct 2010 15:35:59 GMT

The scripts currently used for creating derivatives

←Older revision		Revision as of 15:35, 13 October 2010
Line 18:		Line 18:
	* Linux Perl script that offers to create OCR for all existing transcripts a given collection already online, or of selected transcript numbers across any number of collections: [[Image:moreTranscriptOCR.txt]]		* Linux Perl script that offers to create OCR for all existing transcripts a given collection already online, or of selected transcript numbers across any number of collections: [[Image:moreTranscriptOCR.txt]]

-	* Linux Perl script to create OCR files for items listed in ocrList.txt files located in the /srv/deposits/ocrMe directory, and place those OCR files in the correct web location: [[Image:ocrSelected.txt]]	+	* Linux Perl script to create OCR files for items listed in *ocrList.txt files located in the /srv/deposits/ocrMe directory, and place those OCR files in the correct web location: [[Image:ocrSelected.txt]]

	==OLDER METHODS ARE BELOW==		==OLDER METHODS ARE BELOW==

Jlderidder: /* The scripts currently used for creating derivatives */

Jlderidder — Wed, 13 Oct 2010 14:38:31 GMT

The scripts currently used for creating derivatives

←Older revision		Revision as of 14:38, 13 October 2010
Line 17:		Line 17:

	* Linux Perl script that offers to create OCR for all existing transcripts a given collection already online, or of selected transcript numbers across any number of collections: [[Image:moreTranscriptOCR.txt]]		* Linux Perl script that offers to create OCR for all existing transcripts a given collection already online, or of selected transcript numbers across any number of collections: [[Image:moreTranscriptOCR.txt]]
		+
		+	* Linux Perl script to create OCR files for items listed in ocrList.txt files located in the /srv/deposits/ocrMe directory, and place those OCR files in the correct web location: [[Image:ocrSelected.txt]]

	==OLDER METHODS ARE BELOW==		==OLDER METHODS ARE BELOW==

Scpstu32 at 19:13, 28 June 2010

Scpstu32 — Mon, 28 Jun 2010 19:13:10 GMT

←Older revision		Revision as of 19:13, 28 June 2010
Line 9:		Line 9:

	*These setting were decided upon after consulting the [http://wiki.hydrogenaudio.org/index.php?title=Lame#Portable:_background_noise_and_low_bitrate_requirement.2C_small_sizes\| Hydrogen Audio] site.		*These setting were decided upon after consulting the [http://wiki.hydrogenaudio.org/index.php?title=Lame#Portable:_background_noise_and_low_bitrate_requirement.2C_small_sizes\| Hydrogen Audio] site.
-
-
-

	==The scripts currently used for creating derivatives==		==The scripts currently used for creating derivatives==

Scpstu32 at 19:10, 28 June 2010

Scpstu32 — Mon, 28 Jun 2010 19:10:58 GMT

←Older revision		Revision as of 19:10, 28 June 2010
Line 7:		Line 7:
	* [http://lame.sourceforge.net/ LAME] for some MP3 creation from WAV files. The command used with LAME is of this form:		* [http://lame.sourceforge.net/ LAME] for some MP3 creation from WAV files. The command used with LAME is of this form:
	lame [OLDFILE] [NEWFILE] -V4 --noreplaygain -S		lame [OLDFILE] [NEWFILE] -V4 --noreplaygain -S
		+
		+	*These setting were decided upon after consulting the [http://wiki.hydrogenaudio.org/index.php?title=Lame#Portable:_background_noise_and_low_bitrate_requirement.2C_small_sizes\| Hydrogen Audio] site.
		+

Scpstu32: /* OLDER METHODS ARE BELOW */

Scpstu32 — Mon, 28 Jun 2010 19:00:57 GMT

OLDER METHODS ARE BELOW

←Older revision		Revision as of 19:00, 28 June 2010
Line 20:		Line 20:
	==OLDER METHODS ARE BELOW==		==OLDER METHODS ARE BELOW==

-	''The below information is deprecated; we have replaced the work flow below with[[Moving_Content_to_Acumen_and_Archive]], which enables us to get content online without putting it into the storage archive first. Benefits include that while LOCKSS partners are harvesting our content, we do not have to twiddle our thumbs till they're done; also, by putting the tools into the hands of Digital Services Staff, we free up the programming needs on the server, and avoid bottlenecks.''	+	''The below information is deprecated; we have replaced the work flow below with [[Moving_Content_to_Acumen_and_Archive]], which enables us to get content online without putting it into the storage archive first. Benefits include that while LOCKSS partners are harvesting our content, we do not have to twiddle our thumbs till they're done; also, by putting the tools into the hands of Digital Services Staff, we free up the programming needs on the server, and avoid bottlenecks.''

	----------------------------------------------------------------------------------------------------------------------------		----------------------------------------------------------------------------------------------------------------------------

Jlderidder: /* The scripts currently used for creating derivatives */

Jlderidder — Thu, 10 Jun 2010 15:55:46 GMT

The scripts currently used for creating derivatives

←Older revision		Revision as of 15:55, 10 June 2010
Line 14:		Line 14:
	* Linux Perl script that pulls content from mounted Windows drive, uses ImageMagick to create JPEGs from transcripts, and places them in a directory on the Linux server: [[Image: makeAudioJpegs.txt]]. This also picks up MODS files, MP3 files, and makes OCR of the transcript images (using tesseract-ocr)if a corresponding text file doesn't already exist -- all derivatives go onto the Linux server, and are deleted from the Windows drive.		* Linux Perl script that pulls content from mounted Windows drive, uses ImageMagick to create JPEGs from transcripts, and places them in a directory on the Linux server: [[Image: makeAudioJpegs.txt]]. This also picks up MODS files, MP3 files, and makes OCR of the transcript images (using tesseract-ocr)if a corresponding text file doesn't already exist -- all derivatives go onto the Linux server, and are deleted from the Windows drive.

-	* Linux Perl script that pulls content from mounted Windows drive, uses ImageMagick to create JPEGs from manuscript images, and places them in a directory on the Linux server: [[Image: ~~makeAudioJpegs~~.txt]]	+	* Linux Perl script that pulls content from mounted Windows drive, uses ImageMagick to create JPEGs from manuscript images, and places them in a directory on the Linux server: [[Image: makeJpegs.txt]]. This also picks up MODS files, and makes OCR of the transcript images (using tesseract-ocr)if a corresponding text file doesn't already exist -- all derivatives go onto the Linux server, and are deleted from the Windows drive. Options include the ability to OCR the entire content of images, selected item numbers, and the ability to apply these options to the entire collection already online.
-		+
-		+
-		+
-		+

		+	* Linux Perl script that offers to create OCR for all existing transcripts a given collection already online, or of selected transcript numbers across any number of collections: [[Image:moreTranscriptOCR.txt]]

	==OLDER METHODS ARE BELOW==		==OLDER METHODS ARE BELOW==

Jlderidder at 15:51, 10 June 2010

Jlderidder — Thu, 10 Jun 2010 15:51:42 GMT

←Older revision		Revision as of 15:51, 10 June 2010
Line 1:		Line 1:
		+	==Open source software we are using to support our digital library==

		+	* [http://sourceforge.net/projects/tesseract-ocr/ Tesseract-ocr] for OCR creation. The command used is of this form:
		+	tesseract [OLDFILE] [NEWFILE]
		+	* [http://www.imagemagick.org/ ImageMagick] for JPEG creation from TIFF files. The command used with ImageMagick is of this form (this for the 2048 size):
		+	convert [OLDFILE] -strip -density 96 -resample 96x96 -resize 2048x2048 -filter Cubic -quiet [NEWFILE]
		+	* [http://lame.sourceforge.net/ LAME] for some MP3 creation from WAV files. The command used with LAME is of this form:
		+	lame [OLDFILE] [NEWFILE] -V4 --noreplaygain -S



-	''This page is deprecated; we are replacing it with the [[Moving_Content_to_Acumen_and_Archive]], which enables us to get content online without putting it into the storage archive first. Benefits include that while LOCKSS partners are harvesting our content, we do not have to twiddle our thumbs till they're done; also, by putting the tools into the hands of Digital Services Staff, we free up the programming needs on the server, and avoid bottlenecks.''	+	==The scripts currently used for creating derivatives==

		+	* Linux Perl script that pulls content from mounted Windows drive, uses ImageMagick to create JPEGs from transcripts, and places them in a directory on the Linux server: [[Image: makeAudioJpegs.txt]]. This also picks up MODS files, MP3 files, and makes OCR of the transcript images (using tesseract-ocr)if a corresponding text file doesn't already exist -- all derivatives go onto the Linux server, and are deleted from the Windows drive.

		+	* Linux Perl script that pulls content from mounted Windows drive, uses ImageMagick to create JPEGs from manuscript images, and places them in a directory on the Linux server: [[Image: makeAudioJpegs.txt]]


		+
		+
		+
		+
		+	==OLDER METHODS ARE BELOW==
		+
		+	''The below information is deprecated; we have replaced the work flow below with[[Moving_Content_to_Acumen_and_Archive]], which enables us to get content online without putting it into the storage archive first. Benefits include that while LOCKSS partners are harvesting our content, we do not have to twiddle our thumbs till they're done; also, by putting the tools into the hands of Digital Services Staff, we free up the programming needs on the server, and avoid bottlenecks.''

	----------------------------------------------------------------------------------------------------------------------------		----------------------------------------------------------------------------------------------------------------------------

Jlderidder at 15:57, 4 June 2010

Jlderidder — Fri, 04 Jun 2010 15:57:06 GMT

←Older revision		Revision as of 15:57, 4 June 2010
Line 3:		Line 3:


-		+	''This page is deprecated; we are replacing it with the [[Moving_Content_to_Acumen_and_Archive]], which enables us to get content online without putting it into the storage archive first. Benefits include that while LOCKSS partners are harvesting our content, we do not have to twiddle our thumbs till they're done; also, by putting the tools into the hands of Digital Services Staff, we free up the programming needs on the server, and avoid bottlenecks.''
-	''This ~~section~~ is deprecated; we are replacing it with the [Moving_Content_to_Acumen_and_Archive], which enables us to get content online without putting it into the storage archive first. Benefits include that while LOCKSS partners are harvesting our content, we do not have to twiddle our thumbs till they're done; also, by putting the tools into the hands of Digital Services Staff, we free up the programming needs on the server, and avoid bottlenecks.''	+

Jlderidder at 15:56, 4 June 2010

Jlderidder — Fri, 04 Jun 2010 15:56:38 GMT

←Older revision		Revision as of 15:56, 4 June 2010
Line 1:		Line 1:
-	== Archiving content ==

-	See lines 9-25 and 31-33 on this page: [[Moving_Content_To_Long-Term_Storage]]




-	''~~The following~~ section is deprecated; we are replacing it with the ~~preceding workflow~~, which enables us to get content online without putting it into the storage archive first. Benefits include that while LOCKSS partners are harvesting our content, we do not have to twiddle our thumbs till they're done; also, by putting the tools into the hands of Digital Services Staff, we free up the programming needs on the server, and avoid bottlenecks.''	+	''This section is deprecated; we are replacing it with the [Moving_Content_to_Acumen_and_Archive], which enables us to get content online without putting it into the storage archive first. Benefits include that while LOCKSS partners are harvesting our content, we do not have to twiddle our thumbs till they're done; also, by putting the tools into the hands of Digital Services Staff, we free up the programming needs on the server, and avoid bottlenecks.''
		+
		+

Jlderidder: /* Getting content from the share drive to the storage server to prepare for archiving */

Jlderidder — Fri, 04 Jun 2010 15:54:36 GMT

Getting content from the share drive to the storage server to prepare for archiving

←Older revision		Revision as of 15:54, 4 June 2010
Line 1:		Line 1:
-
-
-
-
-	== Getting content from the share drive to the storage server to prepare for archiving ==
-
-	The moveContent script will copy content to the deposits directory on the storage server, where it will be prepared for archiving at a later date.
-
-	Additionally, it will check the collection-level xml file (in the Admin directory), then add it to the online database which feeds the collection list online. Thus, this script should NOT be run for new collections until the content is indexed (as described in the previous section); otherwise, the link from the collection listing will be dead.
-
-	After making the copy, it will verify that each file copied without alteration, and then delete the copy on the share drive.
-
-	If any files remain in the directories on the share drive, they did NOT copy to the server!! Run the script again, as there may have been a failure of the copy across the network. If this fails, the file will need to be moved manually, and the problem encountered by the script must be resolved.
-
	== Archiving content ==		== Archiving content ==