Sanitize your passwords, please!
Posted on 2022.10.14
I've just stumbled upon a completely weird and obscure error in our Kafka sinks.
2022-10-14T14:19:24.220Z info system/logger.go:74 Initializing connection to host: dev-datawarehouse.internal
2022-10-14T14:19:24.220Z fatal system/logger.go:112 Connection to Clickhouse failed parse "http://sink:[email protected]\[email protected]:8123/default?enable_http_compression=1": net/url: invalid control character in URL
git.dev/sink/system.(*zapLogger).Fatal
/app/sink/system/logger.go:112
git.dev/sink/internals.SetupClickhouseConnection
/app/sink/internals/clickhouse.go:108
git.dev/sink/bootstrap.CreateSink
/app/sink/bootstrap/bootstrap.go:66
main.main
/app/sink/main.go:34
runtime.main
/usr/local/go/src/runtime/proc.go:250
How to approach debugging it?
The key phrase is invalid control character in URL.
So what are the valid characters in URIs?
That StackOverflow answer [1] brings the light to that issue.
Allowed characters as defined by RFC 3986 are ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-._~:/?#[]@!$&'()*+,;=.
So let's see what characters are invalid in our URI:
uri = "http://sink:[email protected]\n@dev-datawarehouse.internal:8123/default?enable_http_compression=1"
allowed_chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-._~:/?#[]@!$&'()*+,;="
for char in allowed_chars:
uri = uri.replace(char, "")
print(uri)
"\n"
Yes, the password contained line break character. Yes, it was generated. You should sanitize your password as usually, password generators are able to generate character sequences that should not be interpreted without sanitization.
| [1] | https://stackoverflow.com/a/1547940/4308541 |