Fix "Broken Pipe" error on SSH connection

Problem

When attempting to use SSH client within a VMWare Player virtual machine, I would receive a a broken pipe error immediately after a successful authentication request.

Same error can be seen in Ansible Job while checking the certificate details

ansible-playbook -i /root/a.yml /root/openshift-ansible/playbooks/openshift-checks/certificate_expiry/easy-mode.yaml -v

failed to connect to the host via ssh mux_client_request_session read from master failed broken pipe

Using the verbose output (ssh -v), I was able to also confirm the authentication was successful before the error. Client and Server both were using different versions of openssh

[root@localhost ~]# ssh -i /root/n.pem USER@hostX -v
OpenSSH_8.0p1, OpenSSL 1.1.1c FIPS 28 May 2019
debug1: Reading configuration data /root/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug1: configuration requests final Match pass
debug1: re-parsing configuration
debug1: Reading configuration data /root/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug1: Connecting to hostX [10.105.11.25] port 22.
debug1: Connection established.
debug1: identity file /root/n.pem type -1
debug1: identity file /root/n.pem-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.0
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4
debug1: match: OpenSSH_7.4 pat OpenSSH_7.0*,OpenSSH_7.1*,OpenSSH_7.2*,OpenSSH_7.3*,OpenSSH_7.4*,OpenSSH_7.5*,OpenSSH_7.6*,OpenSSH_7.7* compat 0x04000002
debug1: Authenticating to hostX:22 as 'itdpl'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: aes256-gcm@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: aes256-gcm@openssh.com MAC: <implicit> compression: none
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:U9mc7XcbBX+zmnv8q6C69spLtor/QOxV/+rMHTIWiIs
debug1: Host 'hostX' is known and matches the ECDSA host key.
debug1: Found key in /root/.ssh/known_hosts:5
debug1: rekey out after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 4294967296 blocks
debug1: Will attempt key: /root/n.pem explicit
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure. Minor code may provide more information
No Kerberos credentials available (default cache: KCM:)


debug1: Unspecified GSS failure. Minor code may provide more information
No Kerberos credentials available (default cache: KCM:)


debug1: Next authentication method: publickey
debug1: Trying private key: /root/n.pem
debug1: Authentication succeeded (publickey).
Authenticated to hostX ([10.105.11.25]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: pledge: network
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug1: Sending environment.
debug1: Sending env LANG = en_US.UTF-8
client_loop: send disconnect: Broken pipe

I had tried some things to edit but it didn't helped

  the sshd/server settings in /etc/ssh/sshd_config :

    TCPKeepAlive yes

    ClientAliveInterval 60

    ClientAliveCountMax 40000

AND

  the ssh/client setting in ~/.ssh/config :

    ServerAliveInterval 60

Fix

The solution is to set the IPQoS option to throughput. You can do it from the command line or in the SSH config file.

SET OPTION VIA CLI

You can set the IPQoS option via the command-line whenever you connect like this:

ssh example.com -o IPQoS=throughput

SET OPTION IN SSH CONFIG

You can set the IPQoS option in the config file (e.g. $HOME/.ssh/config) like this:

# This will apply to all hosts.
Host *    
    IPQoS=throughput