VanDyke Software



This tip from one of our developers explains how to configure VShell® for UNIX (version 2.3 and newer) to use keyboard-interactive authenication using PAM.

Configuring VShell for UNIX to Use Keyboard-Interactive Authentication with PAM

Using VShell 2.3 keyboard-interactive support on UNIX and Linux platforms to provide authentication using PAM requires configuration of VShell and the PAM components provided by the operating system. This overview describes the interaction between VShell and the PAM system, and lays out the core configuration issues for both VShell and PAM. To fully configure PAM-based authentication with VShell, you will want to consult the PAM man pages and other documentation mentioned at the end of this article.

Keyboard-interactive and PAM authentication with VShell

First some basic definitions to get us started:

Keyboard-interactive authentication is a mechanism defined by the Secure Shell (SSH2) protocol that allows for a generic, interactive exchange of messages between an SSH2 server and the SSH2 client that it is attempting to authenticate. As the name of the mechanism implies, the messages exchanged are expected to be textual data entered with a keyboard.

"PAM, or Pluggable Authentication Module, is a UNIX program interface that enables third-party security methods to be used. By using PAM, multiple authentication technologies, such as RSA, DCE, Kerberos, smart card, and S/Key can be added without changing any of the login services, thereby preserving existing system environments." –

One of the motivations for the existence of the keyboard-interactive mechanism is to enable the use of PAM by the Secure Shell protocol on Linux and UNIX systems. Applications such as VShell that are PAM-enabled defer tasks such as authentication, account validation, and password management to software modules that can be configured and changed by the system administrator. As each of these PAM functions is invoked by VShell, any "conversation" that the configured module wants to have with the client is relayed to the SSH2 client by way of the keyboard-interactive mechanism. The client's responses are then delivered back to the PAM module. Finally, if the responses provided to the PAM module meet the requirements of that module, PAM signals VShell that authentication, or some other function, has succeeded. The flexibility that this mechanism provides can be seen in the following sample interactions that an SSH2 client might have with VShell keyboard-interative authentication. Here, the PAM modules that are configured to be used by VShell are those that perform typical password authentication:

    client --> LOGIN "bob"
      "Password:" <-- VShell
    client --> "fizu5ubl"
      SUCCESS <-- VShell

However, on another day, "bob" might experience a more complex conversation when logging into VShell:

    client --> LOGIN "bob"
      "Password:" <-- VShell
    client --> "fizu5ubl"

      "Your password has expired. You must change it now. <-- VShell
      New Password:"

    client --> "eazyone"

      "That's a dictionary word." <-- VShell
      "New Password:"

    client --> "4rt5r7dj"

      "Re-enter new password:" <-- VShell


      SUCCESS <-- VShell

The important point is that the entire conversation, from sending the "Password:" prompt to the dictionary checks against the new password, are controlled by the PAM modules. If an administrator wants VShell to perform another type of authentication, such as a one-time password scheme, it's a simple matter of changing the PAM modules that VShell uses.

Configuring VShell to use keyboard-interactive authentication

To enable users to use keyboard-interactive authentication you need to add it to the "AuthenticationsAllowed" entry in VShell's configuration file (see vshelld_config(5)). By default, VShell does not allow users to authenticate using keyboard-interactive authentication. The following entry in vshelld_config(5) allows users to authenticate using either public key, GSSAPI, or keyboard-interactive authentication:

    AuthenticationsAllowed { publickey, gssapi, keyboard-interactive }

Note: Keyboard-interactive authentication can also be required by adding it to the "AuthenticationsRequired" entry in vshelld_config(5).

Currently, keyboard-interactive authentication can only be configured on UNIX platforms that provide native PAM support.

Configuring PAM for VShell

The primary means of configuring PAM for VShell is to create the VShell-specific PAM configuration that tells PAM which modules are to be used by VShell. Depending on the platform, this may mean that a specific file for VShell needs to be created such as /etc/pam.d/vshelld (Linux, FreeBSD, MAC OS-X). Or, VShell-specific lines need to be added to a common /etc/pam.conf file (Solaris, HP-UX).

The basic PAM configuration contains four types of lines that specify which module is to be used for authentication, account status, password management, and user sessions. An example PAM configuration file for VShell for Linux might look like this:

    auth       required    /lib/security/
    account    required    /lib/security/
    password   required    /lib/security/ retry=3 minlen=10
    session    required    /lib/security/

The "auth" line in this example says to use the pam_unix module for authentication which will result in the user being prompted for their UNIX password in the traditional style. Assuming that the user is able to authenticate, the "account" line requires that the user's account status will be checked in whatever manner the pam_unix module requires. One result of an account check might be that the user is informed that their password has expired, as happened to user "bob" above. And, in order for the user to continue with their session they must first successfully update their password in a manner suitable to the pam_cracklib module. In this case, an administrator has configured the pam_cracklib module to allow the user three attempts to update their password, and to require that passwords have a minimum length of ten characters.

Review and conclusion

This discussion is only intended as an introduction to the ways that PAM can be configured with VShell on most Linux and UNIX systems. While all PAM configurations will look similar to this, the details of which modules are available, what they do, and the arguments that each accepts will vary from system to system. It is important to understand all aspects of PAM when configuring PAM for VShell to ensure that the authentication steps that VShell carries out via its keyboard-interactive authentication mechanism reflect the policies and security requirements of your organization.

On installation, VShell sets up a basic PAM configuration file that uses basic UNIX authentication (password) as in the example above. However, administrators should review that configuration after installation.

There are a number of other aspects of PAM configuration that are not mentioned here, such as the significance of the "required" field shown in the PAM configuration above. For an explanation of that and other details of PAM, a good overview of how PAM works on Linux is available here:

"The Linux-PAM System Administrators' Guide"

For any platform-specific details, be sure to consult the PAM-related man pages and other documentation for each specific UNIX platform.

Further reading:

Sun Microsystems "System Administration Guide", Vol. 2. "Using Authentication Services", section "Introduction to PAM".  

See also man pages for HP-UX, Linux, Solaris and FreeBSD under "pam" and "pam.conf."