If already in deposits - Revision history

Jmanderson: /* TO MOVE CONTENT TO STORAGE */

2012-12-20T19:43:11Z

TO MOVE CONTENT TO STORAGE

Jlderidder: /* TO MOVE CONTENT TO STORAGE */

2012-12-20T17:07:43Z

TO MOVE CONTENT TO STORAGE

←Older revision		Revision as of 17:07, 20 December 2012
Line 112:		Line 112:
	5) do a "chown -R jeremiah:archive" to correct permissions and do a "chmod -R 775 /srv/archive" to protect the archive from unauthorized alteration until next time.		5) do a "chown -R jeremiah:archive" to correct permissions and do a "chmod -R 775 /srv/archive" to protect the archive from unauthorized alteration until next time.

-	6~~) copy current versions of EADs for content out of the archive to notInDbase and process for linking new content.~~	+	6) then go look in the /srv/deposits directory, make sure folders are clean, and delete them.
-	~~Use directories in /srv/deposits for guides as to which collections to link.~~	+
-		+
-	7) then go look in the /srv/deposits directory, make sure folders are clean, and delete them.	+
	I do this: ls /srv/deposits/content/*		I do this: ls /srv/deposits/content/*
	ls /srv/deposits/content//		ls /srv/deposits/content//
Line 124:		Line 121:
	if no content: rm -r /srv/deposits/content/*		if no content: rm -r /srv/deposits/content/*
	(this will recursively delete all content in the directory)		(this will recursively delete all content in the directory)
-		+	7) run ocrSelected ([[Image:ocrSelected.txt]]) in /srv/scripts/surfacing. This will go through the
-	8) run ocrSelected ([[Image:ocrSelected.txt]]) in /srv/scripts/surfacing. This will go through the	+
	*ocrList files in /srv/deposits/ocrMe, select out the items marked with a "1" --		*ocrList files in /srv/deposits/ocrMe, select out the items marked with a "1" --
	locate the tiffs, create OCR files, create directories for them, and place them live in the content directory where they belong.		locate the tiffs, create OCR files, create directories for them, and place them live in the content directory where they belong.

-	9) check results, and delete files in /srv/deposits/ocrMe	+	8) check results, and delete files in /srv/deposits/ocrMe
-		+
-	~~10) Beginning with step 3 of [[EADs]], link new content into EADs.~~	+
-		+
-	~~''The current, working version of these instructions is located here on libcontent1: /srv/scripts/storing/DOTHIS.''~~	+
-		+

	''updated 1 July 2011'' [[User:Jlderidder\|Jlderidder]]		''updated 1 July 2011'' [[User:Jlderidder\|Jlderidder]]

Jlderidder: /* TO MOVE CONTENT TO STORAGE */

2012-12-20T17:05:24Z

TO MOVE CONTENT TO STORAGE

←Older revision		Revision as of 17:05, 20 December 2012
Line 109:		Line 109:
	when it went down, meaning multiple images are corrupted or damaged.		when it went down, meaning multiple images are corrupted or damaged.

-	5) do a "chmod -R ~~755~~ /srv/archive" to protect the archive from unauthorized alteration until next time.	+
		+	5) do a "chown -R jeremiah:archive" to correct permissions and do a "chmod -R 775 /srv/archive" to protect the archive from unauthorized alteration until next time.

	6) copy current versions of EADs for content out of the archive to notInDbase and process for linking new content.		6) copy current versions of EADs for content out of the archive to notInDbase and process for linking new content.

Jlderidder: /* TO MOVE CONTENT TO STORAGE */

2012-12-20T15:54:55Z

TO MOVE CONTENT TO STORAGE

←Older revision		Revision as of 15:54, 20 December 2012
Line 74:		Line 74:
	The output will be written to "nohup.out" -- check it occasionally, and the		The output will be written to "nohup.out" -- check it occasionally, and the
	archive directory for errors. This time it's really copying files into the correct location in the archive (making a		archive directory for errors. This time it's really copying files into the correct location in the archive (making a
-	note of what goes where in '~~moveme~~' for the '~~checkem~~" script in a few steps),	+	note of what goes where in 'moveme2' for the 'checkem2" script in a few steps),
	and it versions metadata and xml files, and it links versioned		and it versions metadata and xml files, and it links versioned
	files and tiffs and wave files into the correct manifests, and links in new collections to the manifest pages of the		files and tiffs and wave files into the correct manifests, and links in new collections to the manifest pages of the
Line 95:		Line 95:
	(be sure to change it back, and then restart the web server)		(be sure to change it back, and then restart the web server)

-	3) then run '~~checkem~~ > look' in /srv/scripts/storing (script available here: [[Image:Checkem.txt]])	+	3) then run 'checkem2 > look' in /srv/scripts/storing (script available here: [[Image:Checkem.txt]])
-	-- it will use the "~~moveme~~" file to	+	-- it will use the "moveme2" file to
	verify md5 sums of content that was moved,		verify md5 sums of content that was moved,
	and delete from the deposits directory if there's a match. If there's NOT a match, it will output		and delete from the deposits directory if there's a match. If there's NOT a match, it will output

Jlderidder: /* TO MOVE CONTENT TO STORAGE */

2012-12-19T20:05:42Z

TO MOVE CONTENT TO STORAGE

Jlderidder: /* TO MOVE CONTENT TO STORAGE */

2012-12-19T20:03:35Z

TO MOVE CONTENT TO STORAGE

←Older revision		Revision as of 20:03, 19 December 2012
Line 68:		Line 68:
	3) Open /srv/scripts/storing/relocating and edit out the line $test = 1;		3) Open /srv/scripts/storing/relocating and edit out the line $test = 1;

-	4) Empty the file /srv/scripts/storing/RelocateManifests, ~~and copy moveme to another filename;~~ delete nohup.out.	+	4) Empty the file /srv/scripts/storing/RelocateManifests, delete nohup.out.

	5) run "nohup relocating" -- this time for real. It will take hours; "nohup" keeps it running even if you disconnect.		5) run "nohup relocating" -- this time for real. It will take hours; "nohup" keeps it running even if you disconnect.

Jlderidder: /* TO MOVE CONTENT TO STORAGE */

2012-12-19T19:41:14Z

TO MOVE CONTENT TO STORAGE

←Older revision		Revision as of 19:41, 19 December 2012
Line 15:		Line 15:
	3) Check to see if the mount point is working. If it is, skip step 4. To check, type in:		3) Check to see if the mount point is working. If it is, skip step 4. To check, type in:
	ls /cifs-mount/		ls /cifs-mount/
-	If nothing is listed, or you get an error, you probably need to do step 4. Run this (#3) twice though,	+	If nothing is listed, or you get an error, you probably need to do step 4. Run this (#3) twice though,
-	because the network is sometimes slow.	+	because the network is sometimes slow.

-	4) To get access to the share drive from the libcontent1 Linux server, run:	+	4) To get access to the share drive from the libcontent1 Linux server, run:
	mount -t cifs -o username=jlderidder,domain=lib //libfs1.lib.ua-net.ua.edu/share/Digital\ Projects/ /cifs-mount		mount -t cifs -o username=jlderidder,domain=lib //libfs1.lib.ua-net.ua.edu/share/Digital\ Projects/ /cifs-mount
	and type in your password to the share drive. (substitute your own username there in the line above)		and type in your password to the share drive. (substitute your own username there in the line above)

Jlderidder: /* TO MOVE CONTENT TO STORAGE */

2012-12-19T19:40:57Z

TO MOVE CONTENT TO STORAGE

←Older revision		Revision as of 19:40, 19 December 2012
Line 13:		Line 13:
	2) To get root access, run: sudo su (if you do not have sudo access, you cannot do this process)		2) To get root access, run: sudo su (if you do not have sudo access, you cannot do this process)

-	3) To get access to the share drive from the libcontent1 Linux server, run:	+	3) Check to see if the mount point is working. If it is, skip step 4. To check, type in:
		+	ls /cifs-mount/
		+	If nothing is listed, or you get an error, you probably need to do step 4. Run this (#3) twice though,
		+	because the network is sometimes slow.
		+
		+	4) To get access to the share drive from the libcontent1 Linux server, run:
	mount -t cifs -o username=jlderidder,domain=lib //libfs1.lib.ua-net.ua.edu/share/Digital\ Projects/ /cifs-mount		mount -t cifs -o username=jlderidder,domain=lib //libfs1.lib.ua-net.ua.edu/share/Digital\ Projects/ /cifs-mount
	and type in your password to the share drive. (substitute your own username there in the line above)		and type in your password to the share drive. (substitute your own username there in the line above)

Jlderidder: /* TO MOVE CONTENT TO STORAGE */

2012-07-12T20:16:26Z

TO MOVE CONTENT TO STORAGE

←Older revision		Revision as of 20:16, 12 July 2012
Line 105:		Line 105:
	5) do a "chmod -R 755 /srv/archive" to protect the archive from unauthorized alteration until next time.		5) do a "chmod -R 755 /srv/archive" to protect the archive from unauthorized alteration until next time.

-	6) copy current versions of EADs for content out of the archive to notInDbase and process for linking new content. Use directories in /srv/deposits for guides as to which collections to link.	+	6) copy current versions of EADs for content out of the archive to notInDbase and process for linking new content.
		+	Use directories in /srv/deposits for guides as to which collections to link.

	7) then go look in the /srv/deposits directory, make sure folders are clean, and delete them.		7) then go look in the /srv/deposits directory, make sure folders are clean, and delete them.
Line 117:		Line 118:
	(this will recursively delete all content in the directory)		(this will recursively delete all content in the directory)

-	8) run ocrSelected ([[Image:ocrSelected.txt]]) in /srv/scripts/surfacing. This will go through the *ocrList files in /srv/deposits/ocrMe, select out the items marked with a "1" --	+	8) run ocrSelected ([[Image:ocrSelected.txt]]) in /srv/scripts/surfacing. This will go through the
		+	*ocrList files in /srv/deposits/ocrMe, select out the items marked with a "1" --
	locate the tiffs, create OCR files, create directories for them, and place them live in the content directory where they belong.		locate the tiffs, create OCR files, create directories for them, and place them live in the content directory where they belong.

Jlderidder at 16:26, 30 May 2012

2012-05-30T16:26:12Z

←Older revision		Revision as of 16:26, 30 May 2012
Line 17:		Line 17:
	and type in your password to the share drive. (substitute your own username there in the line above)		and type in your password to the share drive. (substitute your own username there in the line above)

-	4) Check to see if this collection was already released to LOCKSS harvesters. Run "waitCheck" in /srv/scripts/storing.
-	If a collection has been released, it will tell you when that was, and ask if you want to go ahead and store the new content.
-	If the date is within the past 2-3 weeks, wait. Put the content in another directory if other collections are being stored.
-	If you're going ahead, tell the script yes, and it will copy the existing manifest to another file ending in "_LOCKSS_$date" where $date
-	is today's date. We need this in order to measure the size of our content in LOCKSS, as it also harvests each version of the
-	Manifest.


Line 67:		Line 61:
	and modify to fit).		and modify to fit).

-	3) ~~WARNING! Before running~~ /srv/scripts/storing/relocating~~, do a "chmod -R 755 /srv/archive".~~ ~~This will enable you to~~	+	3) Open /srv/scripts/storing/relocating and edit out the line $test = 1;
-	~~write to~~ the ~~directories as root~~	+

-	4~~) Open /srv/scripts/storing/relocating and edit out the line $test = 1;~~	+	4) Empty the file /srv/scripts/storing/RelocateManifests, and copy moveme to another filename; delete nohup.out.
-		+
-	5) Empty the file /srv/scripts/storing/RelocateManifests, and copy moveme to another filename; delete nohup.out.	+

-	6) run "nohup relocating" -- this time for real. It will take hours; "nohup" keeps it running even if you disconnect.	+	5) run "nohup relocating" -- this time for real. It will take hours; "nohup" keeps it running even if you disconnect.
	The output will be written to "nohup.out" -- check it occasionally, and the		The output will be written to "nohup.out" -- check it occasionally, and the
	archive directory for errors. This time it's really copying files into the correct location in the archive (making a		archive directory for errors. This time it's really copying files into the correct location in the archive (making a

←Older revision		Revision as of 19:43, 20 December 2012
Line 9:		Line 9:
	'''ACCESS and IDENTIFICATION'''		'''ACCESS and IDENTIFICATION'''
	-------------------------		-------------------------
-	1) ssh to ~~libcontent1~~.lib.ua.edu.	+	1) ssh to libcontent.lib.ua.edu.

	2) To get root access, run: sudo su (if you do not have sudo access, you cannot do this process)		2) To get root access, run: sudo su (if you do not have sudo access, you cannot do this process)
Line 18:		Line 18:
	because the network is sometimes slow.		because the network is sometimes slow.

-	4) To get access to the share drive from the ~~libcontent1~~ Linux server, run:	+	4) To get access to the share drive from the libcontent Linux server, run:
	mount -t cifs -o username=jlderidder,domain=lib //libfs1.lib.ua-net.ua.edu/share/Digital\ Projects/ /cifs-mount		mount -t cifs -o username=jlderidder,domain=lib //libfs1.lib.ua-net.ua.edu/share/Digital\ Projects/ /cifs-mount
	and type in your password to the share drive. (substitute your own username there in the line above)		and type in your password to the share drive. (substitute your own username there in the line above)
Line 28:		Line 28:

	1) look through the directories in /srv/deposits/content for anomalies, badly named things,		1) look through the directories in /srv/deposits/content for anomalies, badly named things,
-	and new categories -- any which are not listed on http://~~libcontent1~~.lib.ua.edu/lockss/Manifest.html	+	and new categories -- any which are not listed on http://libcontent.lib.ua.edu/lockss/Manifest.html
	( this manifest is in /srv/www/htdocs/lockss/). Anomalies might be directories which do NOT match		( this manifest is in /srv/www/htdocs/lockss/). Anomalies might be directories which do NOT match
	the expected structure (Admin, Metadata, Scans, Transcripts) or subfolder structure beneath them;		the expected structure (Admin, Metadata, Scans, Transcripts) or subfolder structure beneath them;
Line 90:		Line 90:
	information about new categories to fill this in.		information about new categories to fill this in.

-	2) check links at http://~~libcontent1~~.lib.ua.edu/lockss/Manifest.html -- drill down.	+	2) check links at http://libcontent.lib.ua.edu/lockss/Manifest.html -- drill down.
	If you cannot access it, modify the .htaccess file in /srv/www/htdocs/lockss to allow your IP,		If you cannot access it, modify the .htaccess file in /srv/www/htdocs/lockss to allow your IP,
	and restart the apache web server: '/usr/sbin/apache2ctl restart'		and restart the apache web server: '/usr/sbin/apache2ctl restart'

←Older revision		Revision as of 20:05, 19 December 2012
Line 55:		Line 55:
	1)		1)
	a) open /srv/scripts/storing/relocating (script available here: [[Image:Relocating.txt]]) and set $test = 1.		a) open /srv/scripts/storing/relocating (script available here: [[Image:Relocating.txt]]) and set $test = 1.
-	b) run "relocating > look". It will:	+	b) Remove the file /srv/scripts/storing/RelocateManifests
		+	c) run "relocating > look". It will:
	i) list the files it's going to move, from where and to where in "moveme"		i) list the files it's going to move, from where and to where in "moveme"
	ii) write the new manifests into RelocateManifests so you can see what they're going to look like		ii) write the new manifests into RelocateManifests so you can see what they're going to look like
	iii) write the parent manifests to parentMans, so you can see what they're going to look like		iii) write the parent manifests to parentMans, so you can see what they're going to look like
	iv) write errors to "look" file, as well as other info		iv) write errors to "look" file, as well as other info
-	c) Look at the output in the "look" file, and the manifests concatenated in RelocateManifests and parentMans.	+	d) Look at the output in the "look" file, and the manifests concatenated in RelocateManifests and parentMans.

	2) Make whatever repairs are necessary for script to work. If there are any new categories (z0004?),		2) Make whatever repairs are necessary for script to work. If there are any new categories (z0004?),
Line 68:		Line 69:
	3) Open /srv/scripts/storing/relocating and edit out the line $test = 1;		3) Open /srv/scripts/storing/relocating and edit out the line $test = 1;

-	4) ~~Empty the file /srv/scripts/storing/RelocateManifests,~~ delete nohup.out.	+	4) delete nohup.out.

	5) run "nohup relocating" -- this time for real. It will take hours; "nohup" keeps it running even if you disconnect.		5) run "nohup relocating" -- this time for real. It will take hours; "nohup" keeps it running even if you disconnect.