Web development News

Ten Linux Commands Every Web Developer Should Know

In the early days of the Web, you could build an appealing portfolio with little more than a basic understanding of HTML and a server-side scripting language such as PHP. Although these days the learning curve for developing websites remains remarkably shallow, the industry's growing complexity requires today's professional developer to embrace a variety of technologies. For many developers, this shift hasn't come willfully; in fact, many developers still choose to limit their interaction with the Web server to uploading files via an FTP client. This approach leaves developers unable to manage website assets effectively, monitor server performance, and easily carry out other tasks that are otherwise time-consuming and tedious.
In this article, I'll introduce you to 10 indispensable Linux commands that can make your web development work much easier and even more enjoyable. Whether you prefer to limit your Linux exposure to the occasional SSH session, or have recently switched your desktop operating system to a Linux distribution such as Ubuntu, these 10 commands will hopefully prompt you to begin making your own investigations into the power of the Linux command-line.

Disclaimer for the purists: I refer to "Linux commands" throughout this article merely as a convenience due to the public's widespread general familiarity with the Linux operating system as compared to other variants such as FreeBSD. Of course, these commands will work on a wide variety of operating systems.

1. Retrieving a Software Package

If you need to install some software by retrieving the source package, you can save a step from the process of downloading the package to your laptop and then transferring the file anew via FTP client. Instead, just retrieve the package directly from your web server using the wget command. For instance, to retrieve the latest version of the Zend Framework you can just copy the download link directly from the Zend Framework website and then pass it to the wget command like this:

%>wget http://framework.zend.com/releases/ZendFramework-1.10.3/ZendFramework-1.10.3-minimal.tar.gz

2. Monitoring Server Processes

These days a typical website is powered by much more than just a few HTML pages, often relying on various regularly executing server-side scripts to carry out a variety of maintenance-related tasks. You can keep tabs on your server's executing processes using the top command. This command provides you with a real-time overview of the server's processor activity and memory consumption, listing all processes, the process owner, percentage of CPU used, and duration of execution.
To execute top, just run the command from the command line without any accompanying options (although other options are supported). For instance, the following output was produced after executing the top command on my Ubuntu laptop:

%>top
...
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                      
 2121 wjgilmor  20   0  956m 389m  44m R   66 13.5   1692:22 firefox                                                                      
 1136 root      20   0  126m  42m  16m S   12  1.5  67:22.95 Xorg                                                                         
 7297 wjgilmor  20   0  119m  41m  16m S    4  1.4  25:19.19 filezilla                                                                    
 1919 wjgilmor  20   0 95828 9.8m 7908 S    2  0.3  38:54.57 pulseaudio                                                                   
14159 wjgilmor  20   0  2468 1084  784 R    2  0.0   0:00.02 top                                                                          
    1 root      20   0  2660 1528 1128 S    0  0.1   0:01.05 init

3. Reviewing Log Files

You'll often need to review your server's log files to monitor and resolve any unexpected errors and other issues. Of course, these log files have a tendency to grow as rapidly as your traffic volume. Many novice Linux users will be familiar with the DOS more command, which they use to page through even large log files. Linux offers a far more efficient method for reviewing files, particularly the "end" of files: using the tail command. For instance, to display the last 10 lines of a file just pass the file to the tail command:

%>tail /var/log/apache/error.log

You can use the -n switch to specify a larger number of lines, additionally using the more command to view the output one screen at a time:

%>tail -n 100 /var/log/apache/error.log | more 

To interactively output the newly appended lines as they appear in the file, you can use the -f switch, which will refresh the output every few seconds:

%>tail -f /var/log/apache/error.log

4. Copying Files with scp

An FTP client such as FileZilla works well for transferring files between a developer's laptop and web server, but what about when you need to transfer a file while in the midst of an SSH session? Rather than go through the hassle of logging into the server anew using an FTP client, consider using the scp (secure copy) command. I demonstrated this command in the WDVL.com article "Manage Your Server Accounts Securely with Key-based Authentication," using it to copy my public key to the web server:

%>scp id_rsa.pub webuser@192.168.1.103:/home/webuser/.ssh/id_rsa.pub

In this example, I copy the file id_rsa.pub to the web server identified by the IP address 192.168.1.103 using the server account webuser, and placing the file in the directory /home/webuser/.ssh/.

5. Finding World-writable Files

For security reasons you should always ensure that your website's public-facing files are not world-writable. Otherwise, you could leave your server vulnerable to defacement by an outside party. Hopefully, you configure your file ownership and permissions to disallow such a gaffe from the outset, but it doesn't hurt to regularly audit your server to confirm no such security holes exist. Rather than exhaustively browse the server documents directory, you can instead use the find command to scan a specific directory structure for files configured with certain permissions:

%>find /var/www/ -type f -perm -o+w -exec ls -l {} \;

In this example, I tell find that I'm looking for files (by setting the -type option to f). You can alternatively search for directories by setting -type to d. Finally, I use the -exec option in order to format any found files using the ls command.

6. Backing Up Your Web Directory

Your web hosting provider presumably has implemented a fairly routine backup service. However, I nonetheless encourage you institute your own backup procedure in order to be absolutely certain you'll be able to easily restore your site in the event of a catastrophic server failure. Quite a few solutions exist for backing up your data, among them Amanda, but you can create your own home-brewed solution using the tar command. For instance, to back up the web directory located at /var/www/www.wjgilmore.com while preserving file permissions and excluding the directory /var/www/www.wjgilmore.com/cache/ (as it contains cached files, which are not important for backup purposes), use the following command:

%>tar cpzf www.wjgilmore.com.backup.042710.tgz /var/www.wjgilmore.com

To restore the directory structure, you again use tar, but this time passing the x switch:

%>tar xvpfz www.wjgilmore.com.backup.042710.tgz -C /var/www/

Using tar in conjunction with scp and key-based authentication, you could completely automate the backup process and move the tar file to a remote server!

7. Viewing Your Command History

When getting acquainted with Linux's occasionally esoteric syntax, you'll regularly attempt to recall a particular syntax you executed while attempting to debug a server problem. Rather than continuously referring to bookmarks or other learning resources to recreate the command, you could review your command history using the history command. Executing history will produce a list of the commands you executed. A sample of the output looks like this:

119  more chapters/staging/chapter06.docbook
120  ./convert_program_listings.rb
121  ./convert-chunks.sh 
122  pwd
123  dir
124  more .gitignore 
125  vim .gitignore 
126  git init

Because each command is accompanied by its sequence number, you can easily execute it anew simply by prefacing the sequence number with an exclamation mark, like this:

%>!124
more .gitignore
betas/
cache/
chapters/staging

8. Creating Directory Trees

You'll often need to create a series of nested directories, particularly when starting new projects. Most novice users tediously create each directory by using the mkdir command and then enter each newly created directory only to create the next. You can perform this task in mere seconds using the -p option. For instance, the following example will create a new project directory named webapp, a directory named application inside it, and a directory named controllers inside application:

%>mkdir -p webapp/application/controllers

9. Creating Command Aliases

For whatever reason, I almost always prefer to list directory contents using a format that displays the permissions, owner and group names, size, modification date, and name, by passing the -al option to the ls command:

%>ls -al

Because I use this command so frequently, I've aliased it to something easier to type, namely dir, using the alias command:

%>alias dir='ls -al'

These aliases, however, are lost when you logout of the current session. To make them permanent, you can add them to an account configuration file such as .bashrc.

10. Managing Source Code with git (or svn, or bzr...)

This concluding section isn't so much about a particular command, but rather about one of several possible commands, which perhaps more than any other introduced here you should be well acquainted with if you're not already. The git, svn and bzr commands refer to the command-line interfaces used by the Git, Subversion, and Bazaar version control solutions, respectively. Version control is an essential tool no matter the web project's size and scope, and you're doing both yourself and your clients a significant disservice if you don't rigorously manage projects using one of these powerful solutions.
Even if you don't believe that maintaining a well-documented history of your project's lifecycle is worth the effort, the benefits of version control solutions stretch far beyond mere project management. For instance, you can use basic Git commands to easily deploy and update your website code directly from your development laptop. Although numerous approaches exist, one of the easiest ways is to clone the repository containing your site code to the live server. For instance, if you were using GitHub to host your website repository (call the repository wjgilmore), you could use the following command to retrieve the latest version of your repository:

%>git clone git://github.com/wjgilmore/wjgilmore.git

You can find more sophisticated deployment solutions such as Capistrano, which is regularly used to deploy both Rails and PHP projects.

Conclusion

Linux's command-line capabilities are so vast that this article easily could have covered 1,000 commands rather than a mere 10. Nonetheless, it hopefully was enough to whet your appetite for the incredible power that even occasional users can wield!

About the Author

Jason Gilmore is the founder of EasyPHPWebsites.com. He also is the author of several popular books, including "Easy PHP Websites with the Zend Framework," "Easy PayPal with PHP," and "Beginning PHP and MySQL, Third Edition."
source