OpenCloudOS SSH exec errors

Jun 12, 2024 · 2 min read · 323 Words · -Views -Comments

WebShell SSH login to OpenCloudOS 9 disconnects immediately. After some investigation I found the root cause - here’s a summary.

WebShell SSH login flow

WebShell uses nodejs-ssh2 to implement terminal login:

  1. SSH connect opens an interactive shell for login.
  2. After successful login, the underlying SSH connection is reused to open exec channels to run commands. Here, four commands were executed concurrently.

Error logs

With debug logs enabled, you’ll see something like this:

...
Inbound: CHANNEL_DATA (r:0, 57)
Inbound: CHANNEL_OPEN_CONFIRMATION (r:1, s:1)
Outbound: Sending CHANNEL_REQUEST (r:1, exec: env)
Inbound: CHANNEL_OPEN_CONFIRMATION (r:2, s:2)
Outbound: Sending CHANNEL_REQUEST (r:2, exec: curl https://1991421.cn)
Inbound: CHANNEL_OPEN_CONFIRMATION (r:3, s:3)
Outbound: Sending CHANNEL_REQUEST (r:3, exec: pwd)
Inbound: CHANNEL_OPEN_CONFIRMATION (r:4, s:4)
Outbound: Sending CHANNEL_REQUEST (r:4, exec: id)
Socket ended
Socket closed
/Users/alanhe/GitHub/express-demo/node_modules/ssh2/lib/client.js:1836
      cb(had_err !== true ? had_err : new Error('Unable to exec'));                    ^

As you can see, the connection drops directly, leading to the final error “Unable to exec”.

Analysis

  1. If I disable command execution or keep only one command, it works.

  2. The machine had 1 vCPU and 1 GB RAM. Upgrading to something like 8 GB removed the error.

  3. Reinstalling OpenCloudOS 8 with the same config worked fine.

  4. OpenCloudOS 9 uses OpenSSH v9.3, while OpenCloudOS 8 uses v8.x. Testing other images like Fedora 40 with OpenSSH v9.6 worked fine.

Therefore, I suspect v9 has some limit that causes the connection to drop when too many exec channels are opened. This does not appear to be a general OpenSSH 9.x compatibility issue.

Check server-side OpenSSH version

ssh -V

https://static.1991421.cn/2024/2024-06-13-181408.jpeg

Fixes

  1. Ideally OpenCloudOS 9 should fix this SSH issue - other systems don’t exhibit it.
  2. Until the specific limit is identified, disable or avoid concurrent execs to work around it.

Final Thoughts

While I haven’t pinpointed exactly where the OS enforces this limit, the cause is clear enough. Running commands in parallel isn’t inherently wrong and shouldn’t be a problem, but don’t open too many channels - settings like MaxSessions will still cap things.

Authors
Developer, digital product enthusiast, tinkerer, sharer, open source lover