Skip to content

Commit

Permalink
Make opts use generic (#1628)
Browse files Browse the repository at this point in the history 
* Make opts use generic

* fix glz::auto

* Create opts_internal

* opts_internal enum struct

* Pulling options out of opts

* Use consteval check functions

* use check_hide_non_invocable

* Use auto&&

* check_allow_conversions

* Update json_conformance.cpp

* fix for GCC

* GCC warning fixes

* fix more GCC warnings

* Move validate_skipped out of glz::opts

* Fix GCC warnings

* Create options.md

* Update readme concerning compile time options

* Move shrink_to_fit out of default opts

* Move write_type_info out of default opts

* Update README.md

* Update feature_test.hpp
  • Loading branch information
@stephenberry
stephenberry authored Feb 27, 2025
1 parent e3ccdef commit d5a872b
Show file tree
Hide file tree
Showing 40 changed files with 497 additions and 297 deletions.
38 changes: 18 additions & 20 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ Glaze also supports:
> Glaze `main` branch is currently in development for v5.0.0
>
> v5.0.0 will remove the `detail` namespace from `to/from` specializations and many more internal functions, which will enable cleaner user customization and shorter compilation error messages.
>
> The `glz::opts` struct is also becoming the default options, and more specialized options can be added to custom option structs. See [Options](./docs/options.md) for new compile time option customization. This reduces the size of compiler errors and makes compilation faster.
## With compile time reflection for MSVC, Clang, and GCC!

Expand Down Expand Up @@ -772,7 +774,7 @@ buffer.resize(n);

## Compile Time Options

The `glz::opts` struct defines compile time optional settings for reading/writing.
The `glz::opts` struct defines the default compile time options for reading/writing.

Instead of calling `glz::read_json(...)`, you can call `glz::read<glz::opts{}>(...)` and customize the options.

Expand All @@ -783,13 +785,20 @@ For example: `glz::read<glz::opts{.error_on_unknown_keys = false}>(...)` will tu
- `glz::read<glz::opts{.format = glz::BEVE}>(...)` -> `glz::read_beve(...)`
- `glz::read<glz::opts{.format = glz::JSON}>(...)` -> `glz::read_json(...)`

## Available Compile Time Options
> [!IMPORTANT]
>
> Many options for Glaze are not part of `glz::opts`. This keeps compiler errors shorter and makes options more manageable. See [Options](./docs/options.md) documentation for more details on available compile time options.
### Available Default Compile Time Options

The struct below shows the available options and the default behavior.
The struct below shows the available options in `glz::opts` and the defaults. See [Options](./docs/options.md) for additional options for user customization.

```c++
struct opts {
uint32_t format = json;
struct opts
{
// USER CONFIGURABLE
uint32_t format = JSON;
bool null_terminated = true; // Whether the input buffer is null terminated
bool comments = false; // Support reading in JSONC style comments
bool error_on_unknown_keys = true; // Error when an unknown key is encountered
bool skip_null_members = true; // Skip writing out params in an object if the value is null
Expand All @@ -799,15 +808,13 @@ struct opts {
char indentation_char = ' '; // Prettified JSON indentation char
uint8_t indentation_width = 3; // Prettified JSON indentation size
bool new_lines_in_arrays = true; // Whether prettified arrays should have new lines for each element
bool append_arrays = false; // When reading into an array the data will be appended if the type supports it
bool shrink_to_fit = false; // Shrinks dynamic containers to new size to save memory
bool write_type_info = true; // Write type info for meta objects in variants
bool error_on_missing_keys = false; // Require all non nullable keys to be present in the object. Use
// skip_null_members = false to require nullable members
// skip_null_members = false to require nullable members
bool error_on_const_read =
false; // Error if attempt is made to read into a const value, by default the value is skipped without error
bool validate_skipped = false; // If full validation should be performed on skipped values
bool validate_trailing_whitespace =
false; // If, after parsing a value, we want to validate the trailing whitespace

uint8_t layout = rowwise; // CSV row wise output/input

Expand All @@ -817,22 +824,13 @@ struct opts {
bool bools_as_numbers = false; // Read and write booleans with 1's and 0's

bool quoted_num = false; // treat numbers as quoted or array-like types as having quoted numbers
bool number = false; // read numbers as strings and write these string as numbers
bool number = false; // treats all types like std::string as numbers: read/write these quoted numbers
bool raw = false; // write out string like values without quotes
bool raw_string =
false; // do not decode/encode escaped characters for strings (improves read/write performance)
bool raw_string = false; // do not decode/encode escaped characters for strings (improves read/write performance)
bool structs_as_arrays = false; // Handle structs (reading/writing) without keys, which applies
bool allow_conversions = true; // Whether conversions between convertible types are
// allowed in binary, e.g. double -> float

bool partial_read =
false; // Reads into the deepest structural object and then exits without parsing the rest of the input

// glaze_object_t concepts
bool concatenate = true; // Concatenates ranges of std::pair into single objects when writing

bool hide_non_invocable =
true; // Hides non-invocable members from the cli_menu (may be applied elsewhere in the future)
};
```

Expand Down
67 changes: 67 additions & 0 deletions docs/options.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Compile Time Options

Glaze has a large number of available compile time options. These options are passed as a non-type template parameter to read/write calls.

In order to keep template errors more succinct and allow more customization, Glaze allows users to build their own option structs.

Example:

```c++
struct custom_opts : glz::opts // Inherit from glz::opts to get basic options
{
// Add known Glaze options that do not exist in the base glz::opts
bool validate_trailing_whitespace = true;
};
```

In use:

```c++
// Using custom_opts while specifying a base option (prettify)
glz::write<custom_opts{{glz::opts{.prettify = true}}, true}>(obj);
```
## How Custom Options Work
Glaze uses C++20 concepts to detect if an option exists in the provided options struct, if it doesn't exist, a default value is returned.
```c++
// Code from Glaze demonstrating compile time option check:
consteval bool check_validate_trailing_whitespace(auto&& Opts) {
if constexpr (requires { Opts.validate_trailing_whitespace; }) {
return Opts.validate_trailing_whitespace;
} else {
return false; // Default value (may be different for various options)
}
}
```

## Available Options Outside of glz::opts

In `glaze/core/opts.hpp` you can see the default provided fields in `glz::opts`.

The supported, but not default provided options are listed after `glz::opts` and include:

```c++
bool validate_skipped = false;
// If full validation should be performed on skipped values

bool validate_trailing_whitespace = false;
// If, after parsing a value, we want to validate the trailing whitespace

bool concatenate = true;
// Concatenates ranges of std::pair into single objects when writing

bool allow_conversions = true;
// Whether conversions between convertible types are allowed in binary, e.g. double -> float

bool write_type_info = true;
// Write type info for meta objects in variants

bool shrink_to_fit = false;
// Shrinks dynamic containers to new size to save memory

bool hide_non_invocable = true;
// Hides non-invocable members from the cli_menu (may be applied elsewhere in the future)
```

6 changes: 3 additions & 3 deletions include/glaze/beve/beve_to_json.hpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ namespace glz
{
namespace detail
{
template <opts Opts>
template <auto Opts>
inline void beve_to_json_number(auto&& tag, auto&& ctx, auto&& it, auto&& end, auto& out, auto&& ix) noexcept
{
const auto number_type = (tag & 0b000'11'000) >> 3;
Expand Down Expand Up @@ -104,7 +104,7 @@ namespace glz
}
}

template <glz::opts Opts, class Buffer>
template <auto Opts, class Buffer>
inline void beve_to_json_value(auto&& ctx, auto&& it, auto&& end, Buffer& out, auto&& ix)
{
if (it >= end) [[unlikely]] {
Expand Down Expand Up @@ -637,7 +637,7 @@ namespace glz
}
}

template <glz::opts Opts = glz::opts{}, class BEVEBuffer, class JSONBuffer>
template <auto Opts = glz::opts{}, class BEVEBuffer, class JSONBuffer>
[[nodiscard]] inline error_ctx beve_to_json(const BEVEBuffer& beve, JSONBuffer& out)
{
size_t ix{}; // write index
Expand Down
30 changes: 15 additions & 15 deletions include/glaze/beve/read.hpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,7 @@ namespace glz
constexpr auto is_volatile = std::is_volatile_v<std::remove_reference_t<decltype(value)>>;

if (tag != header) {
if constexpr (Opts.allow_conversions) {
if constexpr (check_allow_conversions(Opts)) {
if constexpr (num_t<T>) {
if ((tag & 0b00000111) != tag::number) {
ctx.error = error_code::syntax_error;
Expand Down Expand Up @@ -728,7 +728,7 @@ namespace glz
if constexpr (resizable<T>) {
value.resize(n);

if constexpr (Opts.shrink_to_fit) {
if constexpr (check_shrink_to_fit(Opts)) {
value.shrink_to_fit();
}
}
Expand Down Expand Up @@ -772,7 +772,7 @@ namespace glz
if constexpr (resizable<T>) {
value.resize(n);

if constexpr (Opts.shrink_to_fit) {
if constexpr (check_shrink_to_fit(Opts)) {
value.shrink_to_fit();
}
}
Expand All @@ -787,9 +787,9 @@ namespace glz
};

if (tag != header) {
if constexpr (Opts.allow_conversions) {
if constexpr (check_allow_conversions(Opts)) {
if (tag != header) [[unlikely]] {
if constexpr (Opts.allow_conversions) {
if constexpr (check_allow_conversions(Opts)) {
if ((tag & 0b00000111) != tag::typed_array) {
ctx.error = error_code::syntax_error;
return;
Expand Down Expand Up @@ -885,7 +885,7 @@ namespace glz
if constexpr (resizable<T>) {
value.resize(n);

if constexpr (Opts.shrink_to_fit) {
if constexpr (check_shrink_to_fit(Opts)) {
value.shrink_to_fit();
}
}
Expand All @@ -902,7 +902,7 @@ namespace glz

x.resize(length);

if constexpr (Opts.shrink_to_fit) {
if constexpr (check_shrink_to_fit(Opts)) {
value.shrink_to_fit();
}

Expand Down Expand Up @@ -948,7 +948,7 @@ namespace glz
if constexpr (resizable<T>) {
value.resize(n);

if constexpr (Opts.shrink_to_fit) {
if constexpr (check_shrink_to_fit(Opts)) {
value.shrink_to_fit();
}
}
Expand All @@ -970,7 +970,7 @@ namespace glz
return;
}
++it;
std::conditional_t<Opts.partial_read, size_t, const size_t> n = int_from_compressed(ctx, it, end);
std::conditional_t<check_partial_read(Opts), size_t, const size_t> n = int_from_compressed(ctx, it, end);
if (bool(ctx.error)) [[unlikely]] {
return;
}
Expand All @@ -982,7 +982,7 @@ namespace glz
if constexpr (resizable<T>) {
value.resize(n);

if constexpr (Opts.shrink_to_fit) {
if constexpr (check_shrink_to_fit(Opts)) {
value.shrink_to_fit();
}
}
Expand All @@ -996,7 +996,7 @@ namespace glz
// for types like std::vector<std::pair...> that can't look up with operator[]
// Instead of hashing or linear searching, we just clear the input and overwrite the entire contents
template <auto Opts>
requires(pair_t<range_value_t<T>> && Opts.concatenate == true)
requires(pair_t<range_value_t<T>> && check_concatenate(Opts) == true)
static void op(auto&& value, is_context auto&& ctx, auto&& it, auto&& end)
{
using Element = typename T::value_type;
Expand All @@ -1011,7 +1011,7 @@ namespace glz
}
const auto tag = uint8_t(*it);
if (tag != header) [[unlikely]] {
if constexpr (Opts.allow_conversions) {
if constexpr (check_allow_conversions(Opts)) {
const auto key_type = tag & 0b000'11'000;
if constexpr (str_t<Key>) {
if (key_type != 0) {
Expand Down Expand Up @@ -1115,7 +1115,7 @@ namespace glz
}
const auto tag = uint8_t(*it);
if (tag != header) [[unlikely]] {
if constexpr (Opts.allow_conversions) {
if constexpr (check_allow_conversions(Opts)) {
const auto key_type = tag & 0b000'11'000;
if constexpr (str_t<Key>) {
if (key_type != 0) {
Expand Down Expand Up @@ -1555,7 +1555,7 @@ namespace glz
return value;
}

template <opts Opts = opts{}, class T>
template <auto Opts = opts{}, class T>
requires(read_supported<BEVE, T>)
[[nodiscard]] inline error_ctx read_file_beve(T& value, const sv file_name, auto&& buffer)
{
Expand Down Expand Up @@ -1591,7 +1591,7 @@ namespace glz
return value;
}

template <opts Opts = opts{}, class T>
template <auto Opts = opts{}, class T>
requires(read_supported<BEVE, T>)
[[nodiscard]] inline error_ctx read_file_beve_untagged(T& value, const std::string& file_name, auto&& buffer)
{
Expand Down
14 changes: 7 additions & 7 deletions include/glaze/beve/skip.hpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ namespace glz
template <>
struct skip_value<BEVE>
{
template <opts Opts>
template <auto Opts>
inline static void op(is_context auto&& ctx, auto&& it, auto&& end) noexcept;
};

Expand Down Expand Up @@ -44,7 +44,7 @@ namespace glz
it += byte_count;
}

template <opts Opts>
template <auto Opts>
inline void skip_object_beve(is_context auto&& ctx, auto&& it, auto&& end) noexcept
{
if (invalid_end(ctx, it, end)) {
Expand Down Expand Up @@ -101,7 +101,7 @@ namespace glz
}
}

template <opts Opts>
template <auto Opts>
inline void skip_typed_array_beve(is_context auto&& ctx, auto&& it, auto&& end) noexcept
{
const auto tag = uint8_t(*it);
Expand Down Expand Up @@ -158,7 +158,7 @@ namespace glz
}
}

template <opts Opts>
template <auto Opts>
inline void skip_untyped_array_beve(is_context auto&& ctx, auto&& it, auto&& end) noexcept
{
++it;
Expand All @@ -172,7 +172,7 @@ namespace glz
}
}

template <opts Opts>
template <auto Opts>
requires(Opts.format == BEVE)
void skip_array(is_context auto&& ctx, auto&& it, auto&& end) noexcept
{
Expand All @@ -190,14 +190,14 @@ namespace glz
}
}

template <opts Opts>
template <auto Opts>
GLZ_ALWAYS_INLINE void skip_additional_beve(is_context auto&& ctx, auto&& it, auto&& end) noexcept
{
++it;
skip_value<BEVE>::op<Opts>(ctx, it, end);
}

template <opts Opts>
template <auto Opts>
inline void skip_value<BEVE>::op(is_context auto&& ctx, auto&& it, auto&& end) noexcept
{
using namespace glz::detail;
Expand Down
Loading

0 comments on commit d5a872b

Please sign in to comment.