Encode unsafe URI characters

[lesha1201]

Currently, nuqs doesn't encode certain characters (e.g. [) in URI which might produce some issues between different environments.

For example, when Safari on iOS identifies [ or { symbol in URI, it automatically encodes whole search parameters even if some parts of it are already encoded. It can cause an issue when you copy a URL containing those characters from some browser and paste it to Safari on iOS. Consider such case

1. User copies a URL https://test.com/?name=[тест. The most of browsers automatically encodes the copied URL and actually copy https://test.com/?name=[%D1%82%D0%B5%D1%81%D1%82 to the clipboard. Notice that it doesn't encode [ symbol.
2. User opens that link (https://test.com/?name=[%D1%82%D0%B5%D1%81%D1%82) on Safari iOS.
3. Safari sees [ character in search params and automatically encodes it (even though some content were already encoded): https://test.com/?name=[%D1%82%D0%B5%D1%81%D1%82 -> https://test.com/?name=%5B%25D1%2582%25D0%25B5%25D1%2581%25D1%2582.
4. User sees [%D1%82%D0%B5%D1%81%D1%82 in UI instead of seeing [тест.

The issue becomes more apparent when you use parseAsJson (because it often produces [, { characters) and UTF-8 characters (e.g. website with non-latin language).

I don't know the actual logic on Safari iOS of encoding the URL. From my quick testing, it always does it if there is at least one [ or { character.

Context

2.4.0

What framework are you using?

• ✅ Next.js (app router)

Which version of your framework are you using?

v15

Reproduction

I've created a minimal example which uses useQueryState (copied from nuqs docs) - https://codesandbox.io/p/devbox/9953r5

1. Open https://9953r5-3000.csb.app/?name=[%D1%82%D0%B5%D1%81%D1%82 (this is the copied version of https://test.com/?name=[тест) on Safari iOS. Instead of having [тест in the input, it shows [%D1%82%D0%B5%D1%81%D1%82. But if you open it, for example, on Chrome or Firefox (not on iOS), then it works fine.

[franky47]

Thanks, I can reproduce the issue when clicking the link (on iOS 18.3, Safari and Firefox, which is all WebKit under the hood), but not when copy/pasting it with the encoded values (https://9953r5-3000.csb.app/?name=[%D1%82%D0%B5%D1%81%D1%82).

I can't reproduce it on Safari on macOS however.

[lesha1201]

@franky47

but not when copy/pasting it with the encoded values (https://9953r5-3000.csb.app/?name=[%D1%82%D0%B5%D1%81%D1%82)

Indeed. Then it's probably not related to WebKit but to something else on iOS. I've also noticed that it has that issue:

1. in Chrome on iOS when using Copied URL suggestion in the search bar
2. when pasting it to the searchbar in Arc browser.

[lesha1201]

@franky47 I was trying to understand how search params should be encoded but there are a lot of uncertainties.

There is one standard which is quite outdated - https://datatracker.ietf.org/doc/html/rfc3986. But that standard doesn't provide details of how we should encode query component but instead provides a general guideline of how to encode all components. By that standard, we should encode all reserved characters (e.g. [).

There is also a new standard - https://url.spec.whatwg.org/, which provides more details of how certain components should be encoded. And by that standard we don't need to encode [ for query component (https://url.spec.whatwg.org/#query-percent-encode-set). But implementation of URLSearchParams uses application/x-www-form-urlencoded percent-encode set which has more character to encode (including [).

Taking all into account, I think it will be more safe to use the same set of code points that should be encoded as URLSearchParams has for now (or just use URLSearchParams for encoding) or at least give an option to opt-out from custom encoding behavior, since that's a native approach for encoding/decoding search params on the web.

[lesha1201]

It seems this issue is related to parsing URL in Swift v6.

https://swiftfiddle.com/?c=H4sIAAAAAAAAA5WRQWvDMAyF%2F4owBBLoEmclsAZKYSuDwU4bO1U7ZKm7GlLZ2EovJf99TtKSHTZo5YuQ36cHTydRi1LogzWO4dm0tK1YG8LwGsXQuuadnaZvD0vYIEEoFHtm68ssWyyKuSvu5lLKtPZfaWVttqLqoJasPKOYXQ9sbiaidR493EdrGT0WQ5%2BPk9ts%2F9%2BC9NmnsDNuSgE0%2FY7kNBrpHZyzCiF9vL3GfvgvJ2lykfZlw4RjFE%2BGgrCtWW17qgTEOBAJimQUd6Aar%2F5CX%2BhYNXrA4GI20me%2FaQdSJ2biGI5cpLlMc9H9AKal%2BZ7zAQAA

UPD: opened issue in Swift - swiftlang/swift-foundation#1190 to better understand what's the right behavior.