`glz::json_t`double serialization, how to customize format?

[acking-you]

Hello @stephenberry,I may have found another bug.

Descreibe my issue

You can see the code below:

int main() {
    glz::json_t v = {{"test", 1.00}, {"test2", 1}};
    std::cout << *glz::write_json(v);
    return 0;
}

Output is that:

{"test":1,"test2":1}

It seems that glz::json_t can't properly serialize the double type?

My current progress

I found that:

1. double type will call this overload:

template <class T>
   requires std::convertible_to<T, val_t> && (!std::derived_from<std::decay_t<T>, json_t>)
json_t(T&& val)
{
   data = val;
}

2. integer type will call this:

template <class T>
   requires std::convertible_to<T, double> && (!std::derived_from<std::decay_t<T>, json_t>) &&
            (!std::convertible_to<T, val_t>)
json_t(T&& val)
{
   data = static_cast<double>(val);
}

3. double type in struct still not work:

struct vv {
    int t;
    double tt;
};

int main() {
    glz::json_t v = {{"test", 1.00}, {"test2", 1}};

    std::cout << *glz::write_json(vv{.t = 1, .tt = 1.0}) << std::endl;
    return 0;
}

4. I found a issue Encoding of double variables that happen to be integral #1592 ,and this question has already been raised.

At last,I noticed that you @stephenberry gave the suggestion to use glz::custom to do this, but this should be difficult to do in std::variant, my current approach is to work around this via glz::raw_json, but this may not be a good solution?

[stephenberry]

The value 1.00 is serialized as 1 because that is the least number of characters needed to roundtrip the value. This is how std::to_chars works along with the default std::format. This will roundtrip fine with Glaze. Is some other library you are using interpreting 1 as an integer?

If you could describe the problem with how 1 is a problem I should be able to give a more helpful response.

If some other library needs a decimal point then we might want to add a std::format like formatting template parameter a type like json_t that allows customization of how numbers are written. But, we first have to figure out what kind of number handling you need.

[acking-you]

If some other library needs a decimal point then we might want to add a std::format like formatting template parameter a type like json_t that allows customization of how numbers are written. But, we first have to figure out what kind of number handling you need.

Thank you for your explanation. I now basically understand why this behavior occurs.
The issue I encountered is: originally, 1.0 is a floating-point number, but since it was parsed as 1, it became an integer, which causes a type error when the upstream receives and parses the JSON data.

I have tried using other JSON libraries, such as RapidJSON or nlohmann JSON, and the outputs were normal. Below is an example of the code using nlohmann JSON:

#include <iostream>
#include "nlohmann/json.hpp"

using json = nlohmann::json;

int main()
{
    json j;

    j["name"] = "Alice";
    j["age"] = 28;
    j["temperature"] = 36.0;

    std::string serialized_str = j.dump();

    std::cout << "Generated JSON:\n" << serialized_str << std::endl;
    return 0;
}

Output is: {"name":"Alice","age":28,"temperature":36.0}

I believe the reason for this result is likely due to the use of std::format, and it seems that nlohmann JSON has implemented its own to_chars method: code

[stephenberry]

glz::json_t doesn't store integers, so are you using your own variant? I don't know what you mean that it was converted into an integer.

In JSON there is no concept of an integer verses a floating point number, just a number. Languages like javascript typically just handle JSON numbers as doubles.

Are you having issues because you're using Glaze together with nlohmann::json and nlohmann::json is parsing your double as an integer? Or, are you using a custom variant in Glaze?

I can see adding an optional to serialize floating point numbers with a decimal point, but I'm still trying to figure out where your issue is coming from. Thanks for walking through this issue with me.

[acking-you]

glz::json_t doesn't store integers, so are you using your own variant? I don't know what you mean that it was converted into an integer.

In JSON there is no concept of an integer verses a floating point number, just a number. Languages like javascript typically just handle JSON numbers as doubles.

What I mean here is that eventually serialization 1.0 becomes 1, which makes our unit test fail, because json parsing written in other languages upstream will report an error because it expects a floating point number

[stephenberry]

Thanks! I was trying to figure out the failure point. Technically Glaze is conformant to the JSON specification by writing out a floating point number as 1, and other JSON libraries will do the same. So, your unit test should probably accept 1 for a floating point input.

However, I can understand cases where you're trying to encode more type information than the JSON specification provides (e.g. whether the number is floating point or an integer). And, where an API might require a decimal point to ensure floating point inputs.

I'll add a compile time option to enable custom floating point serialization, so that you'll get 1.0 rather than 1.

[stephenberry]

I want to use std::format underneath to enable precise formatting for special cases. This requires GCC 12 to be deprecated. GCC 15 is now out, so I'm thinking of making this jump to version 6.0 of Glaze. Glaze tries to always support the latest three versions of GCC, which would be bumped to 13, 14, and 15.