Implement vars, choose/when, and expressions #32

tyler · 2024-12-04T14:13:24Z

Cleanup tasks

revamp the parser to identify expressions in raw text (I think this is ultimately a separate parser from the expression parser. They're similar but not identical.)
identify function calls correctly (i.e. $fn($(var), ...) is valid in raw text ESI contexts
make sure we're interpolating in the right contexts (i.e. inside a vars, when, try, long form assign, etc) and not others
refactor EvalContext and Variables
differentiate interpreter errors (they all come out as Parse errors right now)
make the messages in those interp errors actually useful
make sure errors in the interpreter all bubble out of Value::Error, rather than real errors
- returning a Value::Error should log, but otherwise be interpreted as a Null (I think?!)
fix all todo!() calls

…lds Values, rather than Strings.

…to the EvalContext to differentiate them.

…l needs proper revamp.

…tadata variables.

…ring a Value::Error type.

…InterpolatedContent.

… between lexer and parser clearer anyway.

… a refactor.

…d not parse.

…ch more.

kailan

Really pleased with this, thanks so much both for your hard work and for your patience while I get it reviewed. Just a few areas to address in regards to error handling but I'm happy for this to be released once the printlns are gone.

kailan · 2025-01-06T11:00:36Z

esi/src/parse.rs

 fn do_parse<'a, R>(
    reader: &mut Reader<R>,
    callback: &mut dyn FnMut(Event<'a>) -> Result<()>,
    task: &mut Vec<Event<'a>>,
-    depth: &mut usize,
+    use_queue: bool,
+    try_depth: &mut usize,
+    choose_depth: &mut usize,
    current_arm: &mut Option<TryTagArms>,
    tag: &TagNames,
+    content_type: &ContentType,
 ) -> Result<()>
 where


This function signature is getting pretty complex but we can leave it for now as it's not part of the public API surface and there is a parser rework in progress.

kailan · 2025-01-06T11:01:31Z

esi/src/parse.rs

+        .attributes()
+        .flatten()
+        .find(|attr| attr.key.into_inner() == b"name")
+    {
+        Some(attr) => String::from_utf8(attr.value.to_vec()).unwrap(),
+        None => {
+            return Err(ExecutionError::MissingRequiredParameter(
+                String::from_utf8(elem.name().into_inner().to_vec()).unwrap(),
+                "name".to_string(),
+            ));
+        }
+    };


This parsing pattern appears a few times in this file but, again, we can leave that for the parser rework.

kailan · 2025-01-06T11:02:09Z

esi/src/lib.rs

@@ -504,7 +517,97 @@ fn event_receiver(
                except_task,
            });
        }
-        Event::XML(event) => {
+        Event::ESI(Tag::Assign { name, value }) => {
+            // TODO: the 'name' here might have a subfield, we need to parse it


Is this something that needs to be addressed before release?

kailan · 2025-01-06T11:02:29Z

esi/src/lib.rs

+            }
+        }
+        Event::ESI(Tag::When { .. }) => {
+            println!("Shouldn't be possible to get a When tag here");


Should this throw an error?

If not we should at least avoid println and use the logging macros.

kailan · 2025-01-06T11:02:35Z

esi/src/lib.rs

+                        break;
+                    }
+                } else {
+                    println!("Somehow got something other than a When in a Choose: {when:?}",);


kailan · 2025-01-06T11:03:57Z

esi/src/expression.rs

+}
+
+#[cfg(test)]
+mod tests {


The tests are a large part of this file, I wonder if it's worth extracting them into the tests directory.

Not a blocker

kailan · 2025-01-06T11:05:25Z

esi/src/expression.rs

+) -> Option<Value> {
+    evaluate_interpolated(cur, ctx)
+        .map_err(|e| {
+            println!("Error while evaluating interpolated expression: {e}");


Should this throw an error?
Avoid println

tyler added 9 commits December 4, 2024 11:37

Parse Assign tag

d8cc85a

Parse Vars tag

96dfe51

Short form esi:vars processing works. Added an example for testing.

b04ea81

Basic long form vars works

50f7f68

Basic expression lexer/parser/interpreter which should be easy to expand

2f9f247

Add some TODOs

2bfac00

Clean up some type coercion and make Variables a real struct which ho…

94c25ea

…lds Values, rather than Strings.

Correctly parse choose/when/otherwise

94efa58

Execute choose/when/otherwise tags.

f303107

tyler force-pushed the tyler/vars branch from 16230dd to f303107 Compare December 5, 2024 12:20

tyler added 4 commits December 5, 2024 13:32

Lex matches operators and setup for more

6fff23c

Fix both a parsing bug I introduced as well as one that already existed.

b3eb152

Implement 'matches' operator (as well as ... operators at all)

f74d817

Add regex examples

9530dee

tyler changed the title ~~Implement ESI Vars~~ Implement a vars, choose/when, and expressions Dec 5, 2024

tyler changed the title ~~Implement a vars, choose/when, and expressions~~ Implement vars, choose/when, and expressions Dec 5, 2024

tyler added 5 commits December 5, 2024 17:46

Implement string escaping and triple-tick strings

8bbc600

Function calls

2518381

Implement 'lower' function in library

3b4ed98

Implement html_encode and replace functions. Also integers.

5dd1927

Lex, parse, and scaffold the evaluation of metadata variables.

11926dc

tyler force-pushed the tyler/vars branch from 585e25c to 11926dc Compare December 6, 2024 11:00

tyler added 8 commits December 6, 2024 11:14

Parse subscripted variables

bdf88c4

Changed my mind about meta variables. We'll build the intelligence in…

88c4265

…to the EvalContext to differentiate them.

Store and expose regex captures

54587af

Allow custom match name for captures

2bf3520

Make custom matches in ESI doc work and a demo.

c510f3d

Little tweak to interpolate text parsing to make variables work. Stil…

6e17ba6

…l needs proper revamp.

Refactor EvalContext, adding Request data. Implement basic request me…

bc0039b

…tadata variables.

Implement matches_i operator

bcf717f

tyler added 11 commits December 7, 2024 22:21

Refactor error handling, remove Result from expression execution favo…

e510e86

…ring a Value::Error type.

Rename Event::XML to Event::Content and Event::VarsContent to Event::…

e697141

…InterpolatedContent.

Empty string and 0 should coerce to False. Fix it and test it.

0bb38f9

Refactor lexer and parser to handle {<expr>} case. Makes the division…

c653504

… between lexer and parser clearer anyway.

Expose request metadata

d9e9de9

More correctly identify interpolateable sections. Use logged evaluator.

42b4a2a

'Correctly' identify function calls in raw text. Will circle back for…

b1eaf56

… a refactor.

Fix bug where spaces between args in interpolated function calls woul…

7f539b0

…d not parse.

ESI integers are 32 bits

3e6aea7

Move esi functions into separate file in preparation for adding a bun…

0d13671

…ch more.

Clean up some warnings

22fd90f

tyler marked this pull request as ready for review December 10, 2024 21:34

tyler and others added 3 commits December 10, 2024 13:46

Update README.md with new functionality.

4a55af6

Some cleanup and optimizations

ef9f478

Merge branch 'main' into tyler/vars

2063f4c

tyler force-pushed the tyler/vars branch from c688f03 to 2063f4c Compare December 11, 2024 21:28

vagetman added 3 commits December 11, 2024 20:48

consolidated lex_expr and lex_interpolated_expr, other clean ups

6c2acb1

added tests for lex_interpolated_expr() fn

d4ca54d

enhance error handling and refactor expression evaluation functions

8bd404b

vagetman previously approved these changes Dec 15, 2024

View reviewed changes

Resolve compiler warnings in expression tests

66e6a66

kailan dismissed vagetman’s stale review via 66e6a66 December 16, 2024 14:51

kailan added 2 commits January 6, 2025 10:58

Update authors

2f9e9b6

Update error comment

13d4282

kailan requested changes Jan 6, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement vars, choose/when, and expressions #32

Implement vars, choose/when, and expressions #32

tyler commented Dec 4, 2024 •

edited

Loading

kailan left a comment •

edited

Loading

kailan Jan 6, 2025

kailan Jan 6, 2025

kailan Jan 6, 2025

kailan Jan 6, 2025

kailan Jan 6, 2025

kailan Jan 6, 2025

kailan Jan 6, 2025

kailan Jan 6, 2025

kailan Jan 6, 2025

Implement vars, choose/when, and expressions #32

Are you sure you want to change the base?

Implement vars, choose/when, and expressions #32

Conversation

tyler commented Dec 4, 2024 • edited Loading

kailan left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tyler commented Dec 4, 2024 •

edited

Loading

kailan left a comment •

edited

Loading