Command-line Work on Linux Server

From UA Libraries Digital Services Planning and Documentation
(Difference between revisions)
Jump to: navigation, search
(BASH: to get around on the server, and run scripts)
(6 intermediate revisions by one user not shown)
Line 6: Line 6:
 
#[http://www.gnu.org/software/bash/manual/bashref.html.gz BASH Reference Manual]
 
#[http://www.gnu.org/software/bash/manual/bashref.html.gz BASH Reference Manual]
 
#[http://tigger.uic.edu/depts/accc/software/unixgeneral/vi101.html VI Editor commands]
 
#[http://tigger.uic.edu/depts/accc/software/unixgeneral/vi101.html VI Editor commands]
 
+
#[http://www.linux.ie/newusers/beginners-linux-guide/ps.php Using the PS command]
 +
#[http://linux.about.com/library/cmd/blcmdl_kill.htm  Linux / Unix ps / kill]
  
  
Line 33: Line 34:
 
All commands should be followed by hitting the "enter" key.
 
All commands should be followed by hitting the "enter" key.
  
# 'ls'  means "list."  This is the command you type in to list the contents of the current directory.   
+
# ''''ls''''  means "'''list'''."  This is the command you type in to list the contents of the current directory.   
 
## 'ls -l' (that is el ess space hyphen el) will provide a complete (el is for 'long') listing of all non-system files in a long format, so you can see who the files and directories belong to, permissions issues, size of files, and date created.   
 
## 'ls -l' (that is el ess space hyphen el) will provide a complete (el is for 'long') listing of all non-system files in a long format, so you can see who the files and directories belong to, permissions issues, size of files, and date created.   
 
## To also see system hidden files, use 'ls -la'  (a is for 'all').
 
## To also see system hidden files, use 'ls -la'  (a is for 'all').
 
## To list the contents of a directory, add the name of the subdirectory:  'ls UploadArea' will list the contents of the UploadArea subdirectory.  'ls UploadArea/MODS"  will list the contents of the MODS directory within the UploadArea directory.  In the Linux operating system (as opposed to a Windows operating system) the marker between directory and subdirectory is a forward slash ("/") instead of a backward slash.  
 
## To list the contents of a directory, add the name of the subdirectory:  'ls UploadArea' will list the contents of the UploadArea subdirectory.  'ls UploadArea/MODS"  will list the contents of the MODS directory within the UploadArea directory.  In the Linux operating system (as opposed to a Windows operating system) the marker between directory and subdirectory is a forward slash ("/") instead of a backward slash.  
# 'cd' means "change directory."  This is the command you type in to change directories on the server.   
+
# ''''cd'''' means "'''change directory'''."  This is the command you type in to change directories on the server.   
 
## If you type in 'cd' with no parameters (nothing following the command), you will be returned to your home directory, where you entered the system.
 
## If you type in 'cd' with no parameters (nothing following the command), you will be returned to your home directory, where you entered the system.
 
## If you type in 'cd UploadArea' you will change directories to the UploadArea subdirectory. If then you type in 'ls' you will see the contents of that directory.
 
## If you type in 'cd UploadArea' you will change directories to the UploadArea subdirectory. If then you type in 'ls' you will see the contents of that directory.
Line 45: Line 46:
 
# Viewing file content:
 
# Viewing file content:
 
## To view a file's contents on commandline, type in 'cat filename' where filename is the name of the file you want to view.  If that file is in the sister directory to the directory you are in  -- assume it is in the output directory and you are in the scripts directory -- type in 'cat ../output/filename'.
 
## To view a file's contents on commandline, type in 'cat filename' where filename is the name of the file you want to view.  If that file is in the sister directory to the directory you are in  -- assume it is in the output directory and you are in the scripts directory -- type in 'cat ../output/filename'.
## To view a file, type in 'view filename' where "filename" is the name of the file you want to view.  If you are wanting to edit the file at the command line, type in 'vi filename' instead.  If that file is in the sister directory to the directory you are in -- assuming it is in the output directory and you're in the scripts directory, type in 'vi ../output/filename'.  See the VI section next.
+
## To view a file, type in ''''view filename'''' where "filename" is the name of the file you want to view.  If you are wanting to edit the file at the command line, type in 'vi filename' instead.  If that file is in the sister directory to the directory you are in -- assuming it is in the output directory and you're in the scripts directory, type in 'vi ../output/filename'.  See the VI section next.
  
 
For further information:  [http://www.gnu.org/software/bash/manual/bashref.html.gz BASH Reference Manual]
 
For further information:  [http://www.gnu.org/software/bash/manual/bashref.html.gz BASH Reference Manual]
 +
 +
 +
  
 
==VI:  to view files from the command line on the server==
 
==VI:  to view files from the command line on the server==
# 'view filename'  will open the file in view mode. While it is possible to edit a file in view mode, saving those changes will require the addition of an exclamation point to the save command.
+
 
# Control-f will move the view forward a page at a time
+
# ''''view filename''''  will open the file in view mode. While it is possible to edit a file in view mode, saving those changes will require the addition of an exclamation point to the save command.
# Control-b will move the view backward a page at a time
+
# '''Control-f''' will move the view '''forward''' a page at a time
 +
# '''Control-b''' will move the view '''backward''' a page at a time
 
# arrow keys will scroll  
 
# arrow keys will scroll  
# 'G' takes you to the end of the file (I have no idea what G stands for"ground?")
+
# 'G' means "Go", I think.  If you follow it with a space and number, it will take you to the line in the file corresponding to that number.  By itself, it takes you to the end of the file.  (thanks, Nitin!)
# ':q' (meaning "quit") closes the file.  If you have inadvertantly made edits that you don't want to save, ':q!'  will get you out of the file without saving.  Alternatively, if you want to save your edits, type in ':wq' for "write quit".
+
# '''':q'''' (meaning "'''quit'''") closes the file.  If you have inadvertantly made edits that you don't want to save, '''':q!''''  will get you out of the file '''without saving changes'''.  Alternatively, if you want to save your edits, type in ':wq' for "write quit".
#  Some characters change the mode to where whatever you enter will be typed in, so if you don't want to edit the file, try not to hit any other keys.  If by accident you do get into insertion mode, use Control-Esc to get out of it.  And be sure to use ':q!'  to get out of the file without saving your changes.
+
#  Some characters change the mode to where whatever you enter will be typed in, so if you don't want to edit the file, try not to hit any other keys.  '''If by accident you do get into insertion mode''', use '''Control-Esc''' to get out of it.  And be sure to use '''':q!''''  to get out of the file without saving your changes.
 +
 
 +
 
 +
For further information:  [http://tigger.uic.edu/depts/accc/software/unixgeneral/vi101.html VI Editor commands]
 +
 
 +
 
 +
==PS (Process Status) to find background processes  ==
 +
 
 +
The moveContent and makeJpegs scripts both include a "fork" process. This means that they fork off another process on the system before exiting.  This allows you to close your ssh window and forget about the script, while it goes on working between the servers, sometimes for several hours.
 +
 
 +
However, sometimes things don't work the way you expect, or you find you need to stop and start over for some reason.  In these situations, you need to be able to find out if any processes are running, and you need to be able to stop them (for the latter, see the next section on "kill").
 +
 
 +
''''ps ux''''  will show all the processes currently running by the user who types in this command. For example:
 +
  jlderidder@libcontent1:~> ps ux
 +
  USER      PID %CPU %MEM    VSZ  RSS TTY      STAT START  TIME COMMAND
 +
  1002    10409  0.0  0.0  48848  1944 ?        Ss  10:08  0:00 /usr/lib64/ssh/sftp-server
 +
  1002    11071  1.3  0.0  21416  2532 pts/3    Ss  16:06  0:00 -bash
 +
  1002    11163  0.0  0.0  12960  1048 pts/3    R+  16:06  0:00 ps ux
 +
  1002    32667  0.0  0.0  85640  2016 ?        S    08:04  0:02 sshd: jlderidder@pts/1,pts/2,pts/3               
 +
  1002    32668  0.0  0.0  21416  2532 pts/1    Ss  08:04  0:00 -bash
 +
  1002    32669  0.0  0.0  21416  2536 pts/2    Ss  08:04  0:00 -bash
 +
The above lists me as user #1002, with 6 processes running.  PID is the Process ID Number;  this is the number you need to know in order to stop a process.
 +
In the above listing, PID 10409 is my ssh graphical window, which is often used for transferring files (SFTP means Secure File Transfer Protocol).
 +
I have 3 commandline windows open, listed in PID 32667 (pts 1, 2, and 3);  and in each of those, I'm using a BASH shell.  Each of those is indicated as a separate process, which the server started for me automatically when I opened the terminals.  Those three BASH processes are listed as PIDs 11071, 32668, and 32669.
 +
Also, you'll see that the command "ps ux" that I just typed in, is listed as a process, PID 11163.
 +
 
 +
So now I'm going to pretend I'm Jeremiah, start the makeJpegs script and see what ps prints out:
 +
  jeremiah@libcontent1:/home/jeremiah/UploadArea/scripts> ps ux
 +
  USER      PID %CPU %MEM    VSZ  RSS TTY      STAT START  TIME COMMAND
 +
  jeremiah 10153  0.0  0.0  21288  2428 pts/2    S    16:05  0:00 bash
 +
  jeremiah 11887  0.0  0.0  21288  2424 pts/1    S    16:15  0:00 bash
 +
  jeremiah 11903  0.1  0.0  17152  3708 pts/1    S+  16:15  0:00 /usr/bin/perl ./makeJpegs
 +
  jeremiah 11926  0.0  0.0  12960  1048 pts/2    R+  16:16  0:00 ps ux
 +
 
 +
As you can see, Jeremiah has 2 BASH shells open (PID 10153 and 11887).  The ps command I just typed in shows up as PID 11926.  The makeJpegs script is running as PID 11903.
 +
Under STAT it says "S+".  S means it's sleeping. (R+ means "runnable"; more info here:  [http://www.ahinc.com/linux101/status.htm])  The START value tells me when it was started (16:15 = 4:15 pm).
 +
 
 +
More info about the PS command can be found here:  [http://www.linux.ie/newusers/beginners-linux-guide/ps.php]
 +
 
 +
==Kill (to kill a process located with the PS command above) ==
 +
 
 +
So you've used the PS command and found the PID (process ID) number for the process you want to stop.  How do you stop it?
 +
On the commandline, type in the following, replacing "PIDnum" with the PID number of the process you want to stop:
 +
  '''kill  -9  PIDnum'''
 +
 
 +
That's it.  The "-9" will force the exit of the script, even if it's hung up in an endless loop or something.  You can rerun the ps command above to verify that the script has been killed.  Be aware that you probably have to clean up now, as the script may have been in the middle of writing or copying files at the time you stopped it.  Double check the last several files it wrote (it would be writing quite a few at once) -- or better, delete all it did, and start over.
 +
 
 +
See [http://linux.about.com/library/cmd/blcmdl_kill.htm  Linux / Unix ps / kill] for more information.

Revision as of 16:35, 4 August 2010

The storage/delivery server operating system is SUSE Linux, and the default shell (command line interpreter) via ssh (Secure Shell access) is BASH.

The following are a few commands to help the beginner locate files and open, view, and close them on the server, as well as to run command-line scripts. More information is available at the links provided:

  1. SSH User Manual
  2. BASH Reference Manual
  3. VI Editor commands
  4. Using the PS command
  5. Linux / Unix ps / kill


Contents

SSH: to get to the server

SSH offers 2 interfaces: a command-line interface (the white window icon) and a graphical user interface (GUI: the yellow folder icon). While you may use the GUI for browsing to locate content, you will need the command-line interface for running scripts. So the following instructions are for this latter interface.

  1. Click on the white window icon, or use your computer menu of programs to select the Secure Shell Client.
  2. You may either select the libcontent1 profile, if already programmed in, or click "Quick Connect" and enter in "libcontent1.lib.ua.edu" and the user name.
  3. Enter your password, and click "ok".
  4. You are now at the command line on the server, and within the BASH shell. If at this point, you click on the yellow folder icon at the top, it will open up the graphical user view of your home directory, which may help. Also, clicking on the white window icon at the top will open another command-line interface of your home directory. You may open as many of these as is helpful, so as to work in multiple directories at the same time. This can be useful if you want to run scripts from one directory, view the output in a second directory, and examine the files to be processed in a third directory.
  5. You can resize the windows by dragging their corners, and you can relocate the windows by clicking/dragging the top bar.
  6. to change settings, such as font size, color, type, and background colors, click "edit", click "settings", and select Global Settings/Appearance/Font or Colors. Make your selections and click "OK". It helps if you select text colors and cursor color that will stand out against the background selected.
  7. To exit the server, type "exit" on command line of each window open, or simply close all SSH windows. Before you exit, be sure to close any files you have opened, or the server will retain the "opened" file for recovery until you return.
  8. Never leave your desktop unattended while you have an SSH window open on the server; this is a serious security risk. If you must leave your desktop while SSH is open, lock the access to your desktop first.

For further information: SSH User Manual



BASH: to get around on the server, and run scripts

To find out more about any of these commands, go to Google and type in "manpage" and then the command. Or, on the commandline, type in "man", space, and the basic command without the parameters which follow it. This will open the text version of the manpage stored on the server, and it will open it in VI (next section discussed). For example, "man ls" will give you an option of two versions of the 'ls' command. Enter "1" and a text file will open in the window. Use "control-F" to move forward in the file and "control-B" to move backwards in the file. Alternatively, you can use the arrow keys to scroll up and down. To close the file, type colon q (':q').

All commands should be followed by hitting the "enter" key.

  1. 'ls' means "list." This is the command you type in to list the contents of the current directory.
    1. 'ls -l' (that is el ess space hyphen el) will provide a complete (el is for 'long') listing of all non-system files in a long format, so you can see who the files and directories belong to, permissions issues, size of files, and date created.
    2. To also see system hidden files, use 'ls -la' (a is for 'all').
    3. To list the contents of a directory, add the name of the subdirectory: 'ls UploadArea' will list the contents of the UploadArea subdirectory. 'ls UploadArea/MODS" will list the contents of the MODS directory within the UploadArea directory. In the Linux operating system (as opposed to a Windows operating system) the marker between directory and subdirectory is a forward slash ("/") instead of a backward slash.
  2. 'cd' means "change directory." This is the command you type in to change directories on the server.
    1. If you type in 'cd' with no parameters (nothing following the command), you will be returned to your home directory, where you entered the system.
    2. If you type in 'cd UploadArea' you will change directories to the UploadArea subdirectory. If then you type in 'ls' you will see the contents of that directory.
    3. To move up into the parent directory, type 'cd ..' -- that will take you up one level. 'cd ../..' takes you up 2 levels.
    4. To change to a sister directory that's in the same directory you're in, type 'cd ../sister' where 'sister' is the name of the adjacent directory. Example: 'cd UploadArea/scripts' takes you into the scripts subdirectory of the UploadArea. If then you want to go to the "output" subdirectory of the UploadArea, then type in 'cd ../output'.
  3. To run a script, it is best to change into the directory where that script is located, as often it expects to find the output directory in a relative location to the script. Once you are in the scripts directory, type in the script name and hit enter. If this fails, that means your classpath is not properly set; so report that, but type in './scriptname' instead; that is, preface the script name with './' (period forward slash) to indicate to the BASH shell that you want to run the script located in the directory where you are.
  4. Viewing file content:
    1. To view a file's contents on commandline, type in 'cat filename' where filename is the name of the file you want to view. If that file is in the sister directory to the directory you are in -- assume it is in the output directory and you are in the scripts directory -- type in 'cat ../output/filename'.
    2. To view a file, type in 'view filename' where "filename" is the name of the file you want to view. If you are wanting to edit the file at the command line, type in 'vi filename' instead. If that file is in the sister directory to the directory you are in -- assuming it is in the output directory and you're in the scripts directory, type in 'vi ../output/filename'. See the VI section next.

For further information: BASH Reference Manual



VI: to view files from the command line on the server

  1. 'view filename' will open the file in view mode. While it is possible to edit a file in view mode, saving those changes will require the addition of an exclamation point to the save command.
  2. Control-f will move the view forward a page at a time
  3. Control-b will move the view backward a page at a time
  4. arrow keys will scroll
  5. 'G' means "Go", I think. If you follow it with a space and number, it will take you to the line in the file corresponding to that number. By itself, it takes you to the end of the file. (thanks, Nitin!)
  6. ':q' (meaning "quit") closes the file. If you have inadvertantly made edits that you don't want to save, ':q!' will get you out of the file without saving changes. Alternatively, if you want to save your edits, type in ':wq' for "write quit".
  7. Some characters change the mode to where whatever you enter will be typed in, so if you don't want to edit the file, try not to hit any other keys. If by accident you do get into insertion mode, use Control-Esc to get out of it. And be sure to use ':q!' to get out of the file without saving your changes.


For further information: VI Editor commands


PS (Process Status) to find background processes

The moveContent and makeJpegs scripts both include a "fork" process. This means that they fork off another process on the system before exiting. This allows you to close your ssh window and forget about the script, while it goes on working between the servers, sometimes for several hours.

However, sometimes things don't work the way you expect, or you find you need to stop and start over for some reason. In these situations, you need to be able to find out if any processes are running, and you need to be able to stop them (for the latter, see the next section on "kill").

'ps ux' will show all the processes currently running by the user who types in this command. For example:

 jlderidder@libcontent1:~> ps ux
 USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
 1002     10409  0.0  0.0  48848  1944 ?        Ss   10:08   0:00 /usr/lib64/ssh/sftp-server
 1002     11071  1.3  0.0  21416  2532 pts/3    Ss   16:06   0:00 -bash
 1002     11163  0.0  0.0  12960  1048 pts/3    R+   16:06   0:00 ps ux
 1002     32667  0.0  0.0  85640  2016 ?        S    08:04   0:02 sshd: jlderidder@pts/1,pts/2,pts/3                 
 1002     32668  0.0  0.0  21416  2532 pts/1    Ss   08:04   0:00 -bash
 1002     32669  0.0  0.0  21416  2536 pts/2    Ss   08:04   0:00 -bash

The above lists me as user #1002, with 6 processes running. PID is the Process ID Number; this is the number you need to know in order to stop a process. In the above listing, PID 10409 is my ssh graphical window, which is often used for transferring files (SFTP means Secure File Transfer Protocol). I have 3 commandline windows open, listed in PID 32667 (pts 1, 2, and 3); and in each of those, I'm using a BASH shell. Each of those is indicated as a separate process, which the server started for me automatically when I opened the terminals. Those three BASH processes are listed as PIDs 11071, 32668, and 32669. Also, you'll see that the command "ps ux" that I just typed in, is listed as a process, PID 11163.

So now I'm going to pretend I'm Jeremiah, start the makeJpegs script and see what ps prints out:

 jeremiah@libcontent1:/home/jeremiah/UploadArea/scripts> ps ux
 USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
 jeremiah 10153  0.0  0.0  21288  2428 pts/2    S    16:05   0:00 bash
 jeremiah 11887  0.0  0.0  21288  2424 pts/1    S    16:15   0:00 bash
 jeremiah 11903  0.1  0.0  17152  3708 pts/1    S+   16:15   0:00 /usr/bin/perl ./makeJpegs
 jeremiah 11926  0.0  0.0  12960  1048 pts/2    R+   16:16   0:00 ps ux

As you can see, Jeremiah has 2 BASH shells open (PID 10153 and 11887). The ps command I just typed in shows up as PID 11926. The makeJpegs script is running as PID 11903. Under STAT it says "S+". S means it's sleeping. (R+ means "runnable"; more info here: [1]) The START value tells me when it was started (16:15 = 4:15 pm).

More info about the PS command can be found here: [2]

Kill (to kill a process located with the PS command above)

So you've used the PS command and found the PID (process ID) number for the process you want to stop. How do you stop it? On the commandline, type in the following, replacing "PIDnum" with the PID number of the process you want to stop:

 kill  -9  PIDnum

That's it. The "-9" will force the exit of the script, even if it's hung up in an endless loop or something. You can rerun the ps command above to verify that the script has been killed. Be aware that you probably have to clean up now, as the script may have been in the middle of writing or copying files at the time you stopped it. Double check the last several files it wrote (it would be writing quite a few at once) -- or better, delete all it did, and start over.

See Linux / Unix ps / kill for more information.

Personal tools