Looking up dynamically allocated port numbers to support testing of proxied services

[psychemedia]

Is there a simple pattern for creating basic tests of proxied services, eg using a tool such as playwright?

If we know the port number a service is running on, then a simple test of the following form can be used to check that the page loads with the desired title, and then grabs a screenshot of it:

#test_app.py

PORT= 8765

def test_initial_load(page: Page):
    page.goto(f"http://127.0.0.1:{PORT}")
    expect(page).to_have_title("Page Title")
    page.screenshot(path="proxied-service-screenshot.png")


#Usage: 
# pip install pytest-playwright
# playwright install
# playwright install-deps
# pytest

Using the proxy name alias doesn't seem to get resolved, whereas specifying the port number explicitly does seem to work.

But in a general case, how can we find the port number for a proxied service, particularly if it it allocated dynamically?

I note also that we can get some general metadata about proxied services from http://localhost/server-proxy/servers-info, although not the port numbers.

[takluyver]

Forgive what's probably a stupid question, but can you test it via the proxy URL (e.g. http://127.0.0.1:8888/foo), rather than looking up the port for foo to access it directly?

If JSP launches the server, you need to make at least 1 request via the proxy to ensure it's launched before you try to talk to it. And the server may also know (via some sort of config) what URL it expects to be visible at, so it might not work properly without the proxy URL.

If you do need to access it directly, one option would be to load the same config and entry points that JSP does, and launch services in your test code (at least those using dynamic ports), copying the code to get a random port number:

jupyter-server-proxy/jupyter_server_proxy/handlers.py

Lines 593 to 596 in 5ff4c77

	sock = socket.socket()
	sock.bind(('', self.requested_port))
	self.state['port'] = sock.getsockname()[1]
	sock.close()

[psychemedia]

IIRC, my issue was I have dozens of servers running on different ports, and if I launch a new Jupyter server and to test it, I don't necessarily know what port it's running on if the port is dynamically allocated.

[manics]

Using the proxy name alias doesn't seem to get resolved, whereas specifying the port number explicitly does seem to work.

The proxy name should server:8888/<service-name> work with dynamically allocated ports, we have some tests:

jupyter-server-proxy/tests/resources/jupyter_server_config.py

Lines 31 to 88 in 31e8b30

	c.ServerProxy.servers = {
	'python-http': {
	'command': ['python3', './tests/resources/httpinfo.py', '{port}'],
	},
	'python-http-abs': {
	'command': ['python3', './tests/resources/httpinfo.py', '{port}'],
	'absolute_url': True
	},
	'python-http-port54321': {
	'command': ['python3', './tests/resources/httpinfo.py', '{port}'],
	'port': 54321,
	},
	'python-http-mappath': {
	'command': ['python3', './tests/resources/httpinfo.py', '{port}'],
	'mappath': {
	'/': '/index.html',
	}
	},
	'python-http-mappathf': {
	'command': ['python3', './tests/resources/httpinfo.py', '{port}'],
	'mappath': mappathf,
	},
	'python-websocket' : {
	'command': ['python3', './tests/resources/websocket.py', '--port={port}'],
	'request_headers_override': {
	'X-Custom-Header': 'pytest-23456',
	}
	},
	'python-request-headers': {
	'command': ['python3', './tests/resources/httpinfo.py', '{port}'],
	'request_headers_override': {
	'X-Custom-Header': 'pytest-23456',
	}
	},
	'python-gzipserver': {
	'command': ['python3', './tests/resources/gzipserver.py', '{port}'],
	},
	'python-http-rewrite-response': {
	'command': ['python3', './tests/resources/httpinfo.py', '{port}'],
	'rewrite_response': translate_ciao,
	'port': 54323,
	},
	'python-chained-rewrite-response': {
	'command': ['python3', './tests/resources/httpinfo.py', '{port}'],
	'rewrite_response': [translate_ciao, hello_to_foo],
	},
	'python-cats-only-rewrite-response': {
	'command': ['python3', './tests/resources/httpinfo.py', '{port}'],
	'rewrite_response': [dog_to_cat, cats_only],
	},
	'python-dogs-only-rewrite-response': {
	'command': ['python3', './tests/resources/httpinfo.py', '{port}'],
	'rewrite_response': [cats_only, dog_to_cat],
	},
	'python-proxyto54321-no-command': {
	'port': 54321
	}
	}

https://github.com/jupyterhub/jupyter-server-proxy/blob/main/tests/test_proxies.py

So I think it's worth investigating why it's not working for you.

[takluyver]

IIRC, my issue was I have dozens of servers running on different ports, and if I launch a new Jupyter server and to test it, I don't necessarily know what port it's running on if the port is dynamically allocated.

I didn't quite follow this. Is the issue that you don't know which port either Jupyter or your proxied service is running on?

If your tests know how to talk to the Jupyter server, then as @manics said, the configured proxy name should work, so it's worth figuring out why it doesn't.

[psychemedia]

In https://github.com/jupyterhub/jupyter-server-proxy/blob/main/tests/test_proxies.py, the PORT is set as:

PORT = os.getenv('TEST_PORT', 8888)

If I start a jupyter server, and the default 8888 is already allocated, the server is launched on a new port number. What port number? How do I assign that number to eg the env var TEST_PORT as used in the test script?

[manics]

OK, I think what you're really asking is "how do I find out what port jupyter-server is running on" rather than anything specific to jupyter-server-proxy, since it also applies to any extensions you want to test.

For CI testing I usually aim for as much control over the environment as possible, so in that situation I'd tell jupyter-server which port (jupyter-lab --port=12345 ..., export TEST_PORT=12345 pytest ...) to use rather than letting it pick one itself, same as how the CI tests override the token for convenience:

jupyter-server-proxy/.github/workflows/test.yaml

Lines 81 to 84 in 31e8b30

	JUPYTER_TOKEN=secret jupyter-${{ matrix.jupyter-app }} --config=./tests/resources/jupyter_server_config.py &
	sleep 5
	cd tests
	pytest -k "not acceptance"

If you want to find out the dynamically chosen jupyter-server port though it might be worth asking over on the jupyter-server repo or on Discourse?

[ryanlovett]

For what it's worth, I think the port info could be added to the api at server-proxy/servers-info. The api is only used for UI right now. The endpoint has the comment, "Don't send anything that might be a callable, or leak sensitive info," but it is authenticated, and I'm not sure what is considered sensitive and how it could be leaked.