LINUX Rescue MODE

Rescue Mode

When things go wrong, there are ways to fix problems. However, these methods require that you understand the system well. This chapter will describe the ways that you can boot into rescue mode and single user mode, where you can use your own knowledge to repair the system.

What is Rescue Mode?

Rescue mode provides the ability to boot a small Linux environment entirely from a diskette, CD-ROM, or using some other method.
As the name implies, rescue mode is provided to rescue you from something. During normal operation, your Red Hat Linux system uses files located on your system's hard drive to do everything — run programs, store your files, and more.
However, there may be times when you are unable to get Linux running completely enough to access its files on your system's hard drive. Using rescue mode, you can access the files stored on your system's hard drive, even if you cannot actually run Linux from that hard drive.
Normally, you will need to get into rescue mode for one of two reasons:

• You are unable to boot Linux.
• You are having hardware or software problems, and you want to get a few important files off your system's hard drive.

Next, we will take a closer look at each of these scenarios.

Unable to Boot Linux

This problem is often caused by the installation of another operating system after you have installed Red Hat Linux. Some other operating systems assume that you have no other operating systems on your computer, and they overwrite the Master Boot Record (MBR) that originally contained the GRUB or LILO boot loader. If the boot loader is overwritten in this manner, you will not be able to boot Red Hat Linux unless you can get into rescue mode.
Another common problem is if you use a partitioning tool to resize a partition or create a new partition from free space after installation and it changes the order of your partitions. If the partition number of your / partition changes, the boot loader will not be able to find it to mount the partition. To fix this problem, boot in rescue mode and modify /boot/grub/grub.conf if you are using GRUB or /etc/lilo.conf if you are using LILO.

Hardware/Software Problems

This category includes a wide variety of different situations. Two examples include failing hard drives and forgetting to run LILO after building a new kernel (if you are using LILO as your boot loader). In both of these situations, you may be unable to boot Red Hat Linux. If you can get into rescue mode, you might be able to resolve the problem or at least get copies of your most important files.

Booting Rescue Mode

To boot your system in rescue mode, boot off of a Red Hat Linux boot disk or Red Hat Linux CD-ROM #1, and enter the following command at the installation boot prompt:

boot: linux rescue

You can get to the installation boot prompt in one of these ways:

• By booting your system from an installation boot diskette made from the boot.img image. This method requires that the Red Hat Linux CD-ROM #1 be inserted as the rescue image or that the rescue image be on the hard drive as an ISO image. [1]
• By booting your system from the Red Hat Linux CD-ROM #1.
• By booting from a network disk made from the bootnet.img or PCMCIA boot disk made from pcmcia.img. You can only do this if your network connection is working. You will need to identify the network host and transfer type. For an explanation of how to specify this information, see Installing over the Network in the Official Red Hat Linux Installation Guide.

After booting off a boot disk or Red Hat Linux CD-ROM #1 and providing a valid rescue image, you will see the following message:

The rescue environment will now attempt to find your Red Hat Linux installation and mount it under the directory /mnt/sysimage.  You can then make any changes required to your system.  If you want to proceed with this step choose 'Continue'. If for some reason this process fails you can choose 'Skip' and this step will be skipped and you will go directly to a command shell.

If you select Continue, it will attempt to mount your filesystem under the directory /mnt/sysimage. If it fails to mount a partition, it will notify you. If you select Skip, your filesystem will not be mounted. Choose Skip if you think your filesystem is corrupted.
Once you have your system in rescue mode, a prompt appears on VC (virtual console) 1 and VC 2 (use the [Ctrl]-[Alt]-[F1] key combination to access VC 1 and [Ctrl]-[Alt]-[F2] key combination to access VC 2):

bash#

If you selected Continue to mount your partitions automatically and they were mounted successfully, you are in single-user mode.
To mount a Linux partition manually inside rescue mode, create a directory such as /foo, and type the following command:

mount -t ext3 /dev/hda5 /foo

In the above command, /foo is a directory that you have created and /dev/hda5 is the partition you want to mount. If the partition is of type ext2, replace ext3 with ext2.
If you do not know the names of your partitions, use the following command to list them:

fdisk -l

If your filesystem is mounted and you want to make your system the root partition, use the command chroot /mnt/sysimage. This is useful if you need to run commands such as rpm that require your root partition to be mounted as /. To exit the chroot environment, type exit, and you will return to the bash# prompt.
From the bash# prompt, you can run many useful commands including:

anaconda          gzip        mkfs.ext2   ps badblocks         head        mknod       python bash              hwclock     mkraid      python1.5 cat               ifconfig    mkswap      raidstart chattr            init        mlabel      raidstop chmod             insmod      mmd         rcp chroot            less        mmount      rlogin clock             ln          mmove       rm collage           loader      modprobe    rmmod cp                ls          mount       route cpio              lsattr      mpartition  rpm dd                lsmod       mrd         rsh ddcprobe          mattrib     mread       sed depmode           mbadblocks  mren        sh df                mcd         mshowfat    sync e2fsck            mcopy       mt          tac fdisk             mdel        mtools      tail fsck              mdeltree    mtype       tar fsck.ext2         mdir        mv          touch fsck.ext3         mdu         mzip        traceroute ftp               mformat     open        umount gnome-pty-helper  minfo       pico        uncpio grep              mkdir       ping        uniq gunzip            mke2fs      probe       zcat

Booting Single-User Mode Directly

You may be able to boot single-user mode directly. If your system boots, but does not allow you to log in when it has completed booting, try single-user mode.
If you are using GRUB, use the following steps to boot into single-user mode:

1. If you have a GRUB password configured, type p and enter the password.
2. Select Red Hat Linux with the version of the kernel that you wish to boot and type e for edit. You will be presented with a list of items in the configuration file for the title you just selected.
3. Select the line that starts with kernel and type e to edit the line.
4. Go to the end of the line and type single as a separate word (press the [Spacebar] and then type single). Press [Enter] to exit edit mode.
5. Back at the GRUB screen, type b to boot into single user mode.

If you are using LILO, specify one of these options at the LILO boot prompt (if you are using the graphical LILO, you must press [Ctrl]-[x] to exit the graphical screen and go to the boot: prompt):

boot: linux single boot: linux emergency

In single-user mode, you computer boots to runlevel 1. Your local filesystems will be mounted, but your network will not be activated. You will have a usable system maintenance shell.
In emergency mode, you are booted into the most minimal environment possible. The root filesystem will be mounted read-only and almost nothing will be set up. The main advantage of emergency mode over linux single is that your init files are not loaded. If init is corrupted or not working, you can still mount filesystems to recover data that could be lost during a re-installation.
Have you ever rebuilt a kernel and, eager to try out your new handiwork, rebooted before running /sbin/lilo? If you did not have an entry for an older kernel in lilo.conf, you had a problem. If you would like to know a solution to this problem, read this section.
In many cases, you can boot your Red Hat Linux system from the Red Hat Linux boot disk [1] with your root filesystem mounted and ready to go. Here is how to do it:
Enter the following command at the boot disk's boot: prompt:

linux single root=/dev/hdXX initrd=

Replace the XX in /dev/hdXX with the appropriate letter and number for your root partition.
What does this command do? First, it starts the boot process in single-user mode, with the root partition set to your root partition. The empty initrd specification bypasses the installation-related image on the boot disk, which will cause you to enter single-user mode immediately.
Is there a negative side to using this technique? Unfortunately, yes. Because the kernel on the Red Hat Linux boot disk only has support for IDE built-in, if your system is SCSI-based, you will not be able to do this. In that case, you will have to access rescue mode using the linux rescue command mentioned above.

Step By Step Configuration of syslog server

An important part of maintaining a secure system is keeping track of the activities that take place on the system. If you know what usually happens, such as understanding when users log into your system, you can use log files to spot unusual activity. You can configure what syslogd records through the /etc/syslog.conf configuration file.
The syslogd daemon manages all the logs on your system and coordinates with any of the logging operations of other systems on your network. Configuration information for syslogd is held in the /etc/syslog.conf file, which contains the names and locations for your system log files.
By Default system accept the logs only generated from local host. In this example we will configure a log server and will accept logs from client side.
For this example we are using two systems one linux server one linux clients . To complete these per quest of log server Follow this link
basic network configurations Example and Implementations ip configurations hosts files portmap xinetd services

• A linux server with ip address 192.168.0.254 and hostname Server
• A linux client with ip address 192.168.0.1 and hostname Client1
• Updated /etc/hosts file on both linux system
• Running portmap and xinetd services
• Firewall should be off on server

We suggest you to review that article before start configuration of log server. Once you have completed the necessary steps follow this guide.

Check syslog, portmap, xinetd service in system service it should be on

 #setup Select  System service from list [*]portmap [*]xinetd [*]syslog 

Now restart xinetd and portmap service

To keep on these services after reboot on then via chkconfig command

After reboot verify their status. It must be in running condition

Now open the /etc/sysconfig/syslog file

and locate SYSLOGD_OPTIONS tag

add -r option in this tag to accepts logs from clients

-m 0 disables 'MARK' messages.
-r enables logging from remote machines
-x disables DNS lookups on messages recieved with -r

After saving file restart service with service syslog restart command

On Linux client ping from log server and open /etc/syslog.conf file

Now go to the end of file and do entry for serve as user.* @ [ server IP] as shown in image

After saving file restart service with service syslog restart command

Now restart the client so it can send log entry to server. ( Note that these logs will generate when client boot, so do it restart not shutdown)

 Check clients log on Log server

To check the message of client on server open

In the end of this file you can check the log from clients


=====================================================================
Also we  can use below configuration for setup syslog  Server & Client
=====================================================================

Centralized log server (syslog server)

Suppose we have a server and 5 client machines. And we want to monitor the logs of all those client machines. In situations like this, we will use centralized server as a log server. Whatever events are happening in client machines, the logs will be sent to the server. So that we can monitor all the logs from a centralized server. We make use of syslog service for this.


Features of syslog:
1. Logs the daemon information to localhost
2. Logs the daemon information to Remote host
3. Logs the daemon information to List of users
4. Logs the daemon information to console

Instaltion Package is sysklogd
[root@apache ~]# rpm -q sysklogd
sysklogd-1.4.1-44.el5
[root@apache ~]#

Or you can check as follows:
[root@apache ~]# rpm -qf /etc/syslog.conf
sysklogd-1.4.1-44.el5
[root@apache ~]#

Starting the syslog daemon
[root@apache ~]# /etc/init.d/syslog start
Starting system logger: [ OK ]
Starting kernel logger: [ OK ]
[root@apache ~]#

Checking the process name. it is syslogd
[root@apache ~]# ps -ax | grep syslog
Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.7/FAQ
5190 ? Ss 0:00 syslogd -m 0
5210 pts/0 S+ 0:00 grep syslog
[root@apache ~]#

Configuration of server machine(syslog server)

Service name: syslog
configuration file: /etc/sysconfig/syslog
Steps:
1. Open the /etc/sysconfig/syslog file and add "-r" option to the variable SYSLOGD_OPTIONS as shown below.
[root@server ~]# cat /etc/sysconfig/syslog

# Options to syslogd
# -m 0 disables 'MARK' messages.
# -r enables logging from remote machines
# -x disables DNS lookups on messages recieved with -r
# See syslogd(8) for more details

SYSLOGD_OPTIONS="-r -m 0"

# Options to klogd

# -2 prints all kernel oops messages twice; once for klogd to decode, and
# once for processing with 'ksymoops'
# -x disables all klogd processing of oops messages entirely
# See klogd(8) for more details
KLOGD_OPTIONS="-x"
#
SYSLOG_UMASK=077
# set this to a umask value to use for all log files as in umask(1).
# By default, all permissions are removed for "group" and "other".

[root@server ~]#
2. Restart the syslog service.
[root@server ~]# service syslog restart

Shutting down kernel logger: [ OK ]

Shutting down system logger: [ OK ]

Starting system logger: [ OK ]

Starting kernel logger: [ OK ]

[root@server ~]#

Configuration for client machines

service name: syslog
Configuration file: /etc/syslog.conf

The configuration file /etc/syslog.conf has two parts

Eg:
*.info;mail.none;authpriv.none;cron.none /var/log/messages
[selector field(Facility.priority)]    [action field]
They are selector field and actions field. Selector field is again divided into two. Facilities and priorities.
Facility examples are (authpriv,kern,mail,local7 etc)
The priority is one of the following in ascending order: debug(0), info, notice, warning(warn), error(err), crit, alert, emerg(panic(7))
Actions can be regular files, console, list of users, remote machine ip etc.

Steps:
1. Open the configuration file /etc/syslog.conf and add an entry to redirect the logs to the remote server.

[root@vm1 ~]# cat /etc/syslog.conf

# Log all kernel messages to the console.

# Logging much else clutters up the screen.

#kern.* /dev/console

*.* @192.168.0.19
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none /var/log/messages
# The authpriv file has restricted access.
##authpriv.* /var/log/secure
# Log all the mail messages in one place.
mail.* -/var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg *
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log
[root@vm1 ~]#

2. Restart the service

[root@vm1 ~]# service syslog restart

Shutting down kernel logger: [ OK ]

Shutting down system logger: [ OK ]

Starting system logger: [ OK ]

Starting kernel logger: [ OK ]

[root@vm1 ~]#

Checking:

In server open a terminal and watch /var/log/messages and restart syslog service in client.
root@server ~]# tail -f /var/log/messages

Oct 15 14:42:30 vm1 kernel: Kernel logging (proc) stopped.

Oct 15 14:42:30 vm1 kernel: Kernel log daemon terminating.
Oct 15 14:42:31 vm1 exiting on signal 15
Oct 15 14:42:31 vm1 syslogd 1.4.1: restart.
Oct 15 14:42:31 vm1 kernel: klogd 1.4.1, log source = /proc/kmsg started.

Fields in log from remote machine:

Date Hostname Name_of_the_application: Actual_log_message

If syslog port(514) is not open....
check below steps........

#iptables -A INPUT -m state --state NEW -m udp -p udp --dport 514 -j ACCEPT
#iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 514 -j ACCEPT
#service iptables save

>>and for flushhing iptables us
iptables -F
netstat -an|grep 514
udp        0      0 0.0.0.0:514             0.0.0.0:*

>>Verify that port is open
>>Run following command:
 netstat -tulpn | less
#service syslog restart