Proposal: Method to peek character (like peek for bytes)

[le0pard]

Hello. Thanks for this useful lib

In most cases for building parser/tokenizer on top of strcan, I dont need such method to peek character from some offset without moving position. But while building https://github.com/le0pard/json_mend to repair broken JSON I found that in many cases I need look ahead in string, so can understand with what broken part of a JSON I am dealing with. For now I have such method:

    # Peeks the next character without advancing the scanner
    def peek_char(offset = 0)
      # Handle the common 0-offset case
      if offset.zero?
        # peek(1) returns the next BYTE, not character
        byte_str = @scanner.peek(1)
        return nil if byte_str.empty?

        # Fast path: If it's a standard ASCII char (0-127), return it directly.
        # This avoids the regex overhead for standard JSON characters ({, [, ", etc).
        return byte_str if byte_str.getbyte(0) < 128

        # Slow path: If it's a multibyte char (e.g. “), use regex to match the full character.
        return @scanner.check(/./m)
      end

      # For offsets > 0, we must scan to skip correctly (as characters can be variable width)
      saved_pos = @scanner.pos
      res = nil
      (offset + 1).times do
        res = @scanner.getch
        break if res.nil?
      end
      @scanner.pos = saved_pos
      res
    end

As you can see, I can use check(/./m) to get first character without advancing, but regex is not so fast (that is why even exists this byte_str.getbyte(0) < 128 optimization). For read in some offset I need loop by getch and back original position.

Will be good, if library will have similar method like peek, but which works with characters (name can be peekch).

Thanks

[kou]

Like this?

diff --git a/ext/strscan/strscan.c b/ext/strscan/strscan.c
index c44e77b..9c85037 100644
--- a/ext/strscan/strscan.c
+++ b/ext/strscan/strscan.c
@@ -1136,6 +1136,25 @@ strscan_getch(VALUE self)
                          adjust_register_position(p, p->regs.end[0]));
 }
 
+/*
+ * Peeks at the current character and returns it.
+ *
+ * TODO
+ */
+static VALUE
+strscan_peek_char(VALUE self)
+{
+    struct strscanner *p;
+    long len;
+
+    GET_SCANNER(self, p);
+    if (EOS_P(p))
+        return Qnil;
+
+    len = rb_enc_mbclen(CURPTR(p), S_PEND(p), rb_enc_get(p->str));
+    return extract_range(p, p->curr, p->curr + len);
+}
+
 /*
  * call-seq:
  *   scan_byte -> integer_byte
@@ -2269,6 +2288,7 @@ Init_strscan(void)
     rb_define_method(StringScanner, "get_byte",    strscan_get_byte,    0);
     rb_define_method(StringScanner, "scan_byte",   strscan_scan_byte,   0);
     rb_define_method(StringScanner, "peek",        strscan_peek,        1);
+    rb_define_method(StringScanner, "peek_char",   strscan_peek_char,   0);
     rb_define_method(StringScanner, "peek_byte",   strscan_peek_byte,   0);
 
     rb_define_private_method(StringScanner, "scan_base10_integer", strscan_scan_base10_integer, 0);

[le0pard]

Thanks @kou

If it will support length, like peek, for example peek_char(3) - exactly what needed. Based zero in end of rb_define_method it is implementation without length (also good, but not cover all needs)

But even without length argument - this method will really help

UPD: maybe it can be 2 methods: peek_char and peek_chars(length)

So method will be like this:

def peek_char(offset = 0)
  return @scanner.peek_char if offset.zero?

  @scanner.peek_chars(offset + 1).chars.at(offset)
end

[le0pard]

@kou ping about this new method(s). Thanks

[kou]

peek_char() and peek_chars(length) are acceptable.

I don't want to add peek_char(offset=0) because there is no existing method that processes offset. All of existing methods process the current position.

[le0pard]

Thanks @kou

I think it is strange use case for lib for now, so let's skip