common: make json_parse_input API retry friendly.

The jsmn parser is a beautiful piece of code.  In particular, you can parse
part of a string, then continue where you left off.

We don't take advantage of this, however, meaning for large JSON objects
we parse them multiple times before finally having enough to complete.

Expose the parser state and tokens through the API, so the caller can pass
them in repeatedly.  For the moment, every caller is allocates each time
(except the unit tests).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This commit is contained in:
Rusty Russell
2020-08-20 23:10:32 +09:30
parent f2d0a1d406
commit 3ed03f9c8f
7 changed files with 164 additions and 79 deletions

View File

@@ -1089,21 +1089,25 @@ static void ld_command_handle(struct plugin *plugin,
*/
static bool ld_read_json_one(struct plugin *plugin)
{
bool valid;
const jsmntok_t *toks;
bool complete;
jsmntok_t *toks;
struct command *cmd = tal(plugin, struct command);
jsmn_parser parser;
/* FIXME: This could be done more efficiently by storing the
* toks and doing an incremental parse, like lightning-cli
* does. */
toks = json_parse_input(NULL, plugin->buffer, plugin->used,
&valid);
if (!toks) {
if (!valid) {
plugin_err(plugin, "Failed to parse JSON response '%.*s'",
(int)plugin->used, plugin->buffer);
return false;
}
toks = toks_alloc(NULL);
jsmn_init(&parser);
if (!json_parse_input(&parser, &toks, plugin->buffer, plugin->used,
&complete)) {
tal_free(toks);
plugin_err(plugin, "Failed to parse JSON response '%.*s'",
(int)plugin->used, plugin->buffer);
return false;
}
if (!complete) {
/* We need more. */
return false;
}