Preparing Collections on the S Drive for Online Delivery and Storage - Revision history

Kgmatheny: /* Admin */

Kgmatheny — Tue, 23 Apr 2013 22:38:18 GMT

Admin

←Older revision		Revision as of 22:38, 23 April 2013
Line 58:		Line 58:
	*** '''u0001_0000002.ocrList.txt'''		*** '''u0001_0000002.ocrList.txt'''
	<!--** Finding Aid - .xml extension-->		<!--** Finding Aid - .xml extension-->
-	**Other relevant documents saved as plain .txt (ANSI or UTF-8 without BOM preferred). If possible please incorporate any additional data into the log.txt file. For example, audio collections often have significant item-level notes that we want to retain. These plain text files can be saved with a ".notes.txt" extension ~~- i.e. "~~u0008_0000001_0000001.notes.txt".	+	** [[Skipped Items \| Skipped items list]] - .skipped.txt extension. For batched collections: this should be present ONLY during the last upload, as it will contain information about skipped items across the entire collection.
		+	*** '''u0003_0000193.skipped.txt'''
		+	**Other relevant documents saved as plain .txt (ANSI or UTF-8 without BOM preferred). If possible please incorporate any additional data into the log.txt file. For example, audio collections often have significant item-level notes that we want to retain. These plain text files can be saved with a ".notes.txt" extension
		+	*** '''u0008_0000001_0000001.notes.txt'''

	===Metadata===		===Metadata===

Kgmatheny: changed internal link from TrackingFiles to Tracking Data

Kgmatheny — Mon, 25 Feb 2013 20:56:35 GMT

changed internal link from TrackingFiles to Tracking Data

←Older revision		Revision as of 20:56, 25 February 2013
Line 50:		Line 50:
	***If multiple Digital Collections spawn from the same Analog Collection, there can be more than one Collection Information XML file as follows: u0003_0000001.1.xml, u0003_0000001.2.xml, etc.		***If multiple Digital Collections spawn from the same Analog Collection, there can be more than one Collection Information XML file as follows: u0003_0000001.1.xml, u0003_0000001.2.xml, etc.
	***Make sure to refer to the [[Collection_Information\|Collection_Information]] page regarding acceptable data values.		***Make sure to refer to the [[Collection_Information\|Collection_Information]] page regarding acceptable data values.
-	** '''[[~~TrackingFiles#Scope~~\|u0003_0000001.log.txt]]'''	+	** '''[[Tracking Data\|u0003_0000001.log.txt]]'''
	***Include a text version of the log file with every batch; previous versions of the log will be overwritten by the newest version.		***Include a text version of the log file with every batch; previous versions of the log will be overwritten by the newest version.

Jjcolonnaromano at 21:07, 22 February 2013

Jjcolonnaromano — Fri, 22 Feb 2013 21:07:58 GMT

←Older revision		Revision as of 21:07, 22 February 2013
Line 66:		Line 66:
	***Note the type of spreadsheet is echoed in the segment before the ".txt" -- if this is a batch file, the batch number precedes the m0x value:		***Note the type of spreadsheet is echoed in the segment before the ".txt" -- if this is a batch file, the batch number precedes the m0x value:
	**** example: '''u0002_0000001.1.m01.txt'''.		**** example: '''u0002_0000001.1.m01.txt'''.
		+	**** check this file for: Diacritics, Quotes, UTF-8 encoding
	***see [https://intranet.lib.ua.edu/cataloging/metadata/SpreadsheetRegistry] for more information.		***see [https://intranet.lib.ua.edu/cataloging/metadata/SpreadsheetRegistry] for more information.
	**'''A MODS folder'''		**'''A MODS folder'''

Jjcolonnaromano at 21:04, 22 February 2013

Jjcolonnaromano — Fri, 22 Feb 2013 21:04:38 GMT

←Older revision		Revision as of 21:04, 22 February 2013
Line 65:		Line 65:
	**'''u0003_0000001.m01.txt''' or '''u0002_0000001.m03.txt''' or '''u0008_0000001.m02.txt'''		**'''u0003_0000001.m01.txt''' or '''u0002_0000001.m03.txt''' or '''u0008_0000001.m02.txt'''
	***Note the type of spreadsheet is echoed in the segment before the ".txt" -- if this is a batch file, the batch number precedes the m0x value:		***Note the type of spreadsheet is echoed in the segment before the ".txt" -- if this is a batch file, the batch number precedes the m0x value:
-	***'''u0002_0000001.1.m01.txt'''.	+	**** example: '''u0002_0000001.1.m01.txt'''.
	***see [https://intranet.lib.ua.edu/cataloging/metadata/SpreadsheetRegistry] for more information.		***see [https://intranet.lib.ua.edu/cataloging/metadata/SpreadsheetRegistry] for more information.
	**'''A MODS folder'''		**'''A MODS folder'''
Line 82:		Line 82:
	** u0003_0000001_0000001.txt, u0003_0000002_0001.txt, etc. - plain .txt files corresponding to non-compound and compound objects (inside respective subfolders).		** u0003_0000001_0000001.txt, u0003_0000002_0001.txt, etc. - plain .txt files corresponding to non-compound and compound objects (inside respective subfolders).
	** u0003_0000001_0000001.ocr.txt, u0003_0000002_0001.ocr.txt, etc. - plain .txt files of OCRed tiffs corresponding to non-compound and compound objects (inside respective subfolders). If cleaned up .txt files exist, remove the corresponding .ocr.txt file.		** u0003_0000001_0000001.ocr.txt, u0003_0000002_0001.ocr.txt, etc. - plain .txt files of OCRed tiffs corresponding to non-compound and compound objects (inside respective subfolders). If cleaned up .txt files exist, remove the corresponding .ocr.txt file.
		+
		+	===needsRemediation===
		+	*This folder always exists at the following location.
		+	** S:\Digital Projects\Administrative\Pipeline\collectionInfo\forMDlib\needsRemediation
		+	*Must contain:
		+	**Excel formatted version of the batch metadata.
		+	***example: '''u0002_0000001.1.m01.xlsx'''.

Jjcolonnaromano at 20:12, 22 February 2013

Jjcolonnaromano — Fri, 22 Feb 2013 20:12:34 GMT

←Older revision		Revision as of 20:12, 22 February 2013
Line 64:		Line 64:
	* Must contain:		* Must contain:
	**'''u0003_0000001.m01.txt''' or '''u0002_0000001.m03.txt''' or '''u0008_0000001.m02.txt'''		**'''u0003_0000001.m01.txt''' or '''u0002_0000001.m03.txt''' or '''u0008_0000001.m02.txt'''
-	***Note the type of spreadsheet is echoed in the segment before the ".txt" -- if this is a batch file, the batch number ~~preceeds~~ the m0x value: '''u0002_0000001.1.m01.txt'''.	+	***Note the type of spreadsheet is echoed in the segment before the ".txt" -- if this is a batch file, the batch number precedes the m0x value:
		+	***'''u0002_0000001.1.m01.txt'''.
	***see [https://intranet.lib.ua.edu/cataloging/metadata/SpreadsheetRegistry] for more information.		***see [https://intranet.lib.ua.edu/cataloging/metadata/SpreadsheetRegistry] for more information.
	**'''A MODS folder'''		**'''A MODS folder'''

Jjcolonnaromano at 20:09, 22 February 2013

Jjcolonnaromano — Fri, 22 Feb 2013 20:09:22 GMT

←Older revision		Revision as of 20:09, 22 February 2013
Line 56:		Line 56:
	** [[Thumbs_icons\|Thumbs Icon]] - .png extension.		** [[Thumbs_icons\|Thumbs Icon]] - .png extension.
	** [[OCR List\|OCR list]] - .ocrList.txt extension.		** [[OCR List\|OCR list]] - .ocrList.txt extension.
		+	*** '''u0001_0000002.ocrList.txt'''
	<!--** Finding Aid - .xml extension-->		<!--** Finding Aid - .xml extension-->
	**Other relevant documents saved as plain .txt (ANSI or UTF-8 without BOM preferred). If possible please incorporate any additional data into the log.txt file. For example, audio collections often have significant item-level notes that we want to retain. These plain text files can be saved with a ".notes.txt" extension - i.e. "u0008_0000001_0000001.notes.txt".		**Other relevant documents saved as plain .txt (ANSI or UTF-8 without BOM preferred). If possible please incorporate any additional data into the log.txt file. For example, audio collections often have significant item-level notes that we want to retain. These plain text files can be saved with a ".notes.txt" extension - i.e. "u0008_0000001_0000001.notes.txt".

Jjcolonnaromano at 19:22, 22 February 2013

Jjcolonnaromano — Fri, 22 Feb 2013 19:22:29 GMT

←Older revision		Revision as of 19:22, 22 February 2013
Line 63:		Line 63:
	* Must contain:		* Must contain:
	**'''u0003_0000001.m01.txt''' or '''u0002_0000001.m03.txt''' or '''u0008_0000001.m02.txt'''		**'''u0003_0000001.m01.txt''' or '''u0002_0000001.m03.txt''' or '''u0008_0000001.m02.txt'''
-	***Note the type of spreadsheet is echoed in the segment before the ".txt" -- if this is a batch file, the batch number ~~follows~~ the m0x value: u0002_0000001~~.m01~~.1.txt.	+	***Note the type of spreadsheet is echoed in the segment before the ".txt" -- if this is a batch file, the batch number preceeds the m0x value: '''u0002_0000001.1.m01.txt'''.
	***see [https://intranet.lib.ua.edu/cataloging/metadata/SpreadsheetRegistry] for more information.		***see [https://intranet.lib.ua.edu/cataloging/metadata/SpreadsheetRegistry] for more information.
	**'''A MODS folder'''		**'''A MODS folder'''

Kgmatheny at 15:39, 13 February 2013

Kgmatheny — Wed, 13 Feb 2013 15:39:32 GMT

←Older revision		Revision as of 15:39, 13 February 2013
Line 16:		Line 16:
	# Finalize collection documentation		# Finalize collection documentation
	## '''Tracking Data'''		## '''Tracking Data'''
-	### OLD/SEPARATE FILE: save TrackingFiles spreadsheet (see S:\Digital Projects\Organization\TrackingFiles) as tab delimited .txt file in Admin folder	+	### OLD/SEPARATE FILE: save TrackingFiles spreadsheet (see S:\Digital Projects\Organization\TrackingFiles) as tab delimited .txt file in Admin folder (see [[TrackingFiles]] for more)
-	### NEW/INTEGRATED: COPY Filename column from the metadata spreadsheet to a new sheet, MOVE Tracking Data columns to new sheet, SAVE as tab delimited .txt file in Admin folder	+	### NEW/INTEGRATED: COPY Filename column from the metadata spreadsheet to a new sheet, MOVE Tracking Data columns to new sheet, SAVE as tab delimited .txt file in Admin folder (see [[Tracking Data]] for more)
	## '''OCR List''': see [[OCR List \| these instructions]]		## '''OCR List''': see [[OCR List \| these instructions]]
	## '''Metadata''': save spreadsheet (minus the tracking data you should’ve already removed) as tab delimited .txt file in Metadata folder		## '''Metadata''': save spreadsheet (minus the tracking data you should’ve already removed) as tab delimited .txt file in Metadata folder

Kgmatheny at 17:01, 11 February 2013

Kgmatheny — Mon, 11 Feb 2013 17:01:30 GMT

←Older revision		Revision as of 17:01, 11 February 2013
Line 19:		Line 19:
	### NEW/INTEGRATED: COPY Filename column from the metadata spreadsheet to a new sheet, MOVE Tracking Data columns to new sheet, SAVE as tab delimited .txt file in Admin folder		### NEW/INTEGRATED: COPY Filename column from the metadata spreadsheet to a new sheet, MOVE Tracking Data columns to new sheet, SAVE as tab delimited .txt file in Admin folder
	## '''OCR List''': see [[OCR List \| these instructions]]		## '''OCR List''': see [[OCR List \| these instructions]]
-	## '''Metadata''': save spreadsheet (minus the tracking data you should’ve already removed) as tab delimited .txt file	+	## '''Metadata''': save spreadsheet (minus the tracking data you should’ve already removed) as tab delimited .txt file in Metadata folder
	# [[Making MODS \| Create MODS]]		# [[Making MODS \| Create MODS]]
	# Carefully check contents of the folders (see list below)		# Carefully check contents of the folders (see list below)

Kgmatheny: mucho rearrangement and a few deletions

Kgmatheny — Mon, 11 Feb 2013 17:00:30 GMT

mucho rearrangement and a few deletions

←Older revision		Revision as of 17:00, 11 February 2013
Line 1:		Line 1:
-	~~==Things to do before beginning an~~ upload==	+	The following page assumes the content for upload has already been through the [[Quality_Control \| Quality Control process]]

-	~~The~~ collection number ~~u0003_0000001 will be used~~ as ~~an example~~.	+	==Preparation Procedure==
		+
		+	Choose one of the following checklists:
		+
		+	'''One-Shot Collections''' are small enough to be completed in one batch, so they require no batch numbering or batching process.
		+
		+	'''Ongoing Collections''' are those which will have multiple batches for upload, so they require "batching."
		+
		+	===One-Shot Collections===
		+
		+	# Move collection folder from Digital_Coll_in_progress to Digital_Coll_Complete
		+	# Remove extra text in folder name so that it is labeled with just the collection number
		+	## Example: u0003_0000633_HarperTimetables --> u0003_0000633
		+	# Finalize collection documentation
		+	## '''Tracking Data'''
		+	### OLD/SEPARATE FILE: save TrackingFiles spreadsheet (see S:\Digital Projects\Organization\TrackingFiles) as tab delimited .txt file in Admin folder
		+	### NEW/INTEGRATED: COPY Filename column from the metadata spreadsheet to a new sheet, MOVE Tracking Data columns to new sheet, SAVE as tab delimited .txt file in Admin folder
		+	## '''OCR List''': see [[OCR List \| these instructions]]
		+	## '''Metadata''': save spreadsheet (minus the tracking data you should’ve already removed) as tab delimited .txt file
		+	# [[Making MODS \| Create MODS]]
		+	# Carefully check contents of the folders (see list below)
		+	## Make sure to remove any unnecessary files created during the capture process (for example, test scans or supplementary metadata or text file notes about progress)
		+	# Once everything is okay, you’re ready to [[Most Content \| Upload Content]]!

-	===~~Check Subfolders~~ of ~~Collection Level Folder~~===	+	===Ongoing Collections===
		+
		+	# Set up collection folder in Digital_Coll_Complete, and create inside it
		+	## Admin
		+	## Metadata
		+	## Transcripts (if necessary)
		+	# Move Scans folder from ongoing collection folder in Digital_Coll_in_progress to this new collection folder in Digital_Coll_Complete
		+	# Follow the procedures for [[Batches \| Creating Batch Documentation]]
		+	# [[Making MODS \| Create MODS]]
		+	# Double-check contents of the folders (see list below)
		+	# Once everything is okay, you’re ready to [[Most Content \| Upload Content]]!
		+
		+
		+	==Checking Folders==
	The Collection Level folder contains [[Share_Drive_Protocols#Contents\|subfolders]] and their content must adhere to certain specifications prior to the collection being considered ready to "ship" for online access and long term storage.		The Collection Level folder contains [[Share_Drive_Protocols#Contents\|subfolders]] and their content must adhere to certain specifications prior to the collection being considered ready to "ship" for online access and long term storage.

-	====Admin====	+	The collection number u0003_0000001 will be used as an example.
		+
		+	===Admin===
	* This folder must exist.		* This folder must exist.

	*Must contain:		*Must contain:
	** '''[[Collection_Information\|u0003_0000001.xml]]'''		** '''[[Collection_Information\|u0003_0000001.xml]]'''
		+	***If multiple Digital Collections spawn from the same Analog Collection, there can be more than one Collection Information XML file as follows: u0003_0000001.1.xml, u0003_0000001.2.xml, etc.
	***Make sure to refer to the [[Collection_Information\|Collection_Information]] page regarding acceptable data values.		***Make sure to refer to the [[Collection_Information\|Collection_Information]] page regarding acceptable data values.
	** '''[[TrackingFiles#Scope\|u0003_0000001.log.txt]]'''		** '''[[TrackingFiles#Scope\|u0003_0000001.log.txt]]'''
Line 21:		Line 59:
	**Other relevant documents saved as plain .txt (ANSI or UTF-8 without BOM preferred). If possible please incorporate any additional data into the log.txt file. For example, audio collections often have significant item-level notes that we want to retain. These plain text files can be saved with a ".notes.txt" extension - i.e. "u0008_0000001_0000001.notes.txt".		**Other relevant documents saved as plain .txt (ANSI or UTF-8 without BOM preferred). If possible please incorporate any additional data into the log.txt file. For example, audio collections often have significant item-level notes that we want to retain. These plain text files can be saved with a ".notes.txt" extension - i.e. "u0008_0000001_0000001.notes.txt".

-	* If multiple Digital Collections spawn from the same Analog Collection, there can be more than one Collection Information XML file as follows:	+	===Metadata===
-	** u0003_0000001.1.xml	+
-	** u0003_0000001.2.xml	+
-		+
-		+
-	====Metadata====	+
	* This folder must exist.		* This folder must exist.
	* Must contain:		* Must contain:
	**'''u0003_0000001.m01.txt''' or '''u0002_0000001.m03.txt''' or '''u0008_0000001.m02.txt'''		**'''u0003_0000001.m01.txt''' or '''u0002_0000001.m03.txt''' or '''u0008_0000001.m02.txt'''
		+	***Note the type of spreadsheet is echoed in the segment before the ".txt" -- if this is a batch file, the batch number follows the m0x value: u0002_0000001.m01.1.txt.
		+	***see [https://intranet.lib.ua.edu/cataloging/metadata/SpreadsheetRegistry] for more information.
		+	**'''A MODS folder'''
		+	*** This folder will contain all the MODS files created via Archivist Utility (see: [[Making MODS]]).

-	~~(Note the type of spreadsheet is echoed in the segment before the ".txt" -- if this is a batch file, the batch number follows the m0x value: u0002_0000001.m01.1.txt.)~~	+	===Scans===
-		+
-	~~(see [https://intranet.lib.ua.edu/cataloging/metadata/SpreadsheetRegistry] for more information.)~~	+
-		+
-		+
-	***This is tab-delimited text export of the original spreadsheet. (If this file was exported out of Excel as a tab-delimited txt file, you must open the file with Notepad ++ and do a Search and Replace to remove the quotation marks that have been inserted at each tab.)	+
-	***The source .xlsx spreadsheet should be moved to S:\Digital Projects\Administrative\Pipeline\collectionInfo\forMDlib\needsRemediation.	+
-		+
-	<blockquote>''If this is a large or ongoing collection, the tab-delimited text export should contain ONLY the metadata for the items currently being transported to storage. The text file itself should have a period and then a number to indicate which portion, or "batch", of the complete metadata this is. The first tab-delimited export would be named, for example, u0003_0000001.1.txt, and would contain the first 500 entries, for example. The second tab-delimited export, for items 501-1000, would be named u0003_0000001.m01.2.txt, and so forth. Thus, only by collecting all these tab-delimited exports do we have a complete set of descriptive metadata for the collection items. For more on how to parse these "batches" out from the complete set of descriptive metadata, see [[Parsing Metadata]].''</blockquote>	+
-		+
-	*'''A MODS folder'''	+
-	** This folder will contain all the MODS files created via Archivist Utility (see: [[Making MODS]]).	+
-		+
-	====Scans====	+
	* This folder must exist.		* This folder must exist.

	*Must only contain:		*Must only contain:
	**'''Scans (tiffs/wavs)''' of non-compound objects and compound objects (inside respective subfolders). All other files types will not be retained. Temporary files and thumbs.db files do not have to be deleted since they will be removed upon transfer to Storage.		**'''Scans (tiffs/wavs)''' of non-compound objects and compound objects (inside respective subfolders). All other files types will not be retained. Temporary files and thumbs.db files do not have to be deleted since they will be removed upon transfer to Storage.
-

	====Transcripts====		====Transcripts====
-	* This folder ~~must~~ exist only if [[transcripts]] exist.	+	* This folder can exist only if [[transcripts]] exist.
	*Must only contain one or more of the following types of files:		*Must only contain one or more of the following types of files:
	** u0003_0000001_0000001.tif, u0003_0000002_0001.tif, etc. - corresponding to non-compound and compound objects (inside respective subfolders).		** u0003_0000001_0000001.tif, u0003_0000002_0001.tif, etc. - corresponding to non-compound and compound objects (inside respective subfolders).
	** u0003_0000001_0000001.txt, u0003_0000002_0001.txt, etc. - plain .txt files corresponding to non-compound and compound objects (inside respective subfolders).		** u0003_0000001_0000001.txt, u0003_0000002_0001.txt, etc. - plain .txt files corresponding to non-compound and compound objects (inside respective subfolders).
	** u0003_0000001_0000001.ocr.txt, u0003_0000002_0001.ocr.txt, etc. - plain .txt files of OCRed tiffs corresponding to non-compound and compound objects (inside respective subfolders). If cleaned up .txt files exist, remove the corresponding .ocr.txt file.		** u0003_0000001_0000001.ocr.txt, u0003_0000002_0001.ocr.txt, etc. - plain .txt files of OCRed tiffs corresponding to non-compound and compound objects (inside respective subfolders). If cleaned up .txt files exist, remove the corresponding .ocr.txt file.
-
-
-	===Perform Quality Control Tasks===
-	see [[Quality_Control]]
-
-	Note: quality control tasks should have already have been performed by scanning technicians during the QC process, but it's a good idea to run the QC scripts again. It takes very little time and helps to catch any mistakes that might have gotten through.
-
-
-	===Spot check all .xml, .txt, and .xlsx files===
-	* Check all such files for proper filenames and extensions.
-	* Open all such files and look for anomalies and inconsistencies, misspellings, and missing data, etc.
-	**Ideally, no additional fields such as "Notes" are in the Metadata file. "Notes" as such should be deleted or moved to the appropriate row in the log.txt file.
-	*** Make sure the Format column in the Metadata file has not been altered to the Time format. If a tab delimited metadata file is opened via Excel (especially by right clicking the file and choosing to open in Excel), the format column if like: 3 p., 4 p., etc. Will get interpreted as 3:00 PM, 4:00 PM, etc. If then resaved as .txt, times will have been saved instead of page #s. The way around this is to have Excel open first, choose Open. Open your text file and while you are being interrogated by Excel about how to import, set the Format column as "Text".
-
-	Obviously, if errors are found after text exports and MODS files are made, then the Excel file needs to be corrected and the text and MODS files remade.
-
-
-	===Check all Folder names===
-	*Make sure folders are named correctly and that there are no superfluous word concatenations to object level folders, etc.
-
-
-	===Moving the Files to the Server===
-	The folder should now be prepared to run scripts and place the files on the storage server.
-
-	see: [http://www.lib.ua.edu/wiki/digcoll/index.php/Most_Content Most_Content]