Skip to content

Protolib #3808

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
cthulhu-rider wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Protolib #3808

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5c99975
internal: Add protobuf parsers
cthulhu-rider Feb 9, 2026
1935f0b
internal/object: Add header and parent header protobuf parsers
cthulhu-rider Feb 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
169 changes: 168 additions & 1 deletion internal/object/wire.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ import (
"errors"
"fmt"
"io"
"strconv"

iprotobuf "github.com/nspcc-dev/neofs-node/internal/protobuf"
"github.com/nspcc-dev/neofs-sdk-go/object"
protoobject "github.com/nspcc-dev/neofs-sdk-go/proto/object"
"github.com/nspcc-dev/neofs-sdk-go/proto/refs"
Expand All @@ -19,8 +21,31 @@ const (
fieldObjectSignature
fieldObjectHeader
fieldObjectPayload

FieldHeaderVersion = 1
Copy link
Member

@roman-khimov roman-khimov Feb 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any way to avoid these redefinitions? Like some variable initialization in init() via https://pkg.go.dev/reflect#StructTag (SDK proto packages have this data) or some autogenerator?

Copy link
Contributor Author

@cthulhu-rider cthulhu-rider Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they are declared in SDK, we can expose them. Codegen does not have them. Reflection is an overkill to me. So, const declaration is the easiest way to me now

Copy link
Member

@roman-khimov roman-khimov Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certainly we don't want another copy of this set. Then ideally it should be autogenerated in SDK as well, but that's a different story.

FieldHeaderContainerID = 2
FieldHeaderOwnerID = 3
FieldHeaderCreationEpoch = 4
FieldHeaderPayloadLength = 5
FieldHeaderPayloadHash = 6
FieldHeaderType = 7
FieldHeaderHomoHash = 8
FieldHeaderSessionToken = 9
FieldHeaderAttributes = 10
FieldHeaderSplit = 11
FieldHeaderSessionTokenV2 = 12

FieldHeaderSplitParent = 1
FieldHeaderSplitPrevious = 2
FieldHeaderSplitParentSignature = 3
FieldHeaderSplitParentHeader = 4
FieldHeaderSplitChildren = 5
FieldHeaderSplitSplitID = 6
FieldHeaderSplitFirst = 7
)

var errEmptyData = errors.New("empty data")

// WriteWithoutPayload writes the object header to the given writer without the payload.
func WriteWithoutPayload(w io.Writer, obj object.Object) error {
header := obj.CutPayload().Marshal()
Expand All @@ -41,7 +66,7 @@ func ExtractHeaderAndPayload(data []byte) (*object.Object, []byte, error) {
)

if len(data) == 0 {
return nil, nil, fmt.Errorf("empty data")
return nil, nil, errEmptyData
}

for offset < len(data) {
Expand Down Expand Up @@ -108,3 +133,145 @@ func ReadHeaderPrefix(r io.Reader) (*object.Object, []byte, error) {
}
return ExtractHeaderAndPayload(buf[:n])
}

// GetNonPayloadFieldBounds seeks ID, signature and header in object message and
// parses their boundaries.
//
// If buf is empty, GetNonPayloadFieldBounds returns an error.
//
// If any field is missing, no error is returned.
//
// Message should have ascending field order, otherwise error returns.
func GetNonPayloadFieldBounds(buf []byte) (iprotobuf.FieldBounds, iprotobuf.FieldBounds, iprotobuf.FieldBounds, error) {
var idf, sigf, hdrf iprotobuf.FieldBounds
if len(buf) == 0 {
return idf, sigf, hdrf, errEmptyData
}

var off int
var prevNum protowire.Number
loop:
for {
num, typ, n, err := iprotobuf.ParseTag(buf[off:])
Copy link
Contributor

@End-rey End-rey Feb 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If len(buf)==0, here we have an error? Maybe we can do a check at the beginning of the function?

Copy link
Contributor Author

@cthulhu-rider cthulhu-rider Feb 13, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i dont expect this to be called with empty buffer. If so, this is an error to me

Copy link
Member

@roman-khimov roman-khimov Feb 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This depends on use, if used by FSTree directly it can be problematic, empty buffer can appear as a result of a file read (touch fstree/a/b/c/dobj), erroring out is better than panic in this case.

Copy link
Contributor Author

@cthulhu-rider cthulhu-rider Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added check

if err != nil {
return idf, sigf, hdrf, err
}

if num > fieldObjectHeader {
break
}
if num < prevNum {
return idf, sigf, hdrf, iprotobuf.NewUnorderedFieldsError(prevNum, num)
}
if num == prevNum {
return idf, sigf, hdrf, iprotobuf.NewRepeatedFieldError(num)
}
prevNum = num

f, err := iprotobuf.ParseLENFieldBounds(buf, off, n, num, typ)
if err != nil {
return idf, sigf, hdrf, err
}

switch num {
case fieldObjectID:
idf = f
case fieldObjectSignature:
sigf = f
case fieldObjectHeader:
hdrf = f
break loop
default:
panic("unreachable with num " + strconv.Itoa(int(num)))
}

off = f.To

if off == len(buf) {
break
}
}

return idf, sigf, hdrf, nil
}

// GetParentNonPayloadFieldBounds seeks parent's ID, signature and header in child
// object message and parses their boundaries.
//
// If buf is empty, GetParentNonPayloadFieldBounds returns an error.
//
// If any field is missing, no error is returned.
//
// Message should have ascending field order, otherwise error returns.
func GetParentNonPayloadFieldBounds(buf []byte) (iprotobuf.FieldBounds, iprotobuf.FieldBounds, iprotobuf.FieldBounds, error) {
var idf, sigf, hdrf iprotobuf.FieldBounds
if len(buf) == 0 {
return idf, sigf, hdrf, errEmptyData
}

rootHdrf, err := iprotobuf.GetLENFieldBounds(buf, fieldObjectHeader)
if err != nil {
return idf, sigf, hdrf, err
}

if rootHdrf.IsMissing() {
return idf, sigf, hdrf, nil
}

splitf, err := iprotobuf.GetLENFieldBounds(buf[rootHdrf.ValueFrom:rootHdrf.To], FieldHeaderSplit)
if err != nil {
return idf, sigf, hdrf, err
}

if splitf.IsMissing() {
return idf, sigf, hdrf, nil
}

buf = buf[:rootHdrf.ValueFrom+splitf.To]
off := rootHdrf.ValueFrom + splitf.ValueFrom
var prevNum protowire.Number
loop:
for {
num, typ, n, err := iprotobuf.ParseTag(buf[off:])
if err != nil {
return idf, sigf, hdrf, err
}

if num > FieldHeaderSplitParentHeader {
break
}
if num < prevNum {
return idf, sigf, hdrf, iprotobuf.NewUnorderedFieldsError(prevNum, num)
}
if num == prevNum {
return idf, sigf, hdrf, iprotobuf.NewRepeatedFieldError(num)
}
prevNum = num

f, err := iprotobuf.ParseLENFieldBounds(buf, off, n, num, typ)
if err != nil {
return idf, sigf, hdrf, err
}

switch num {
case FieldHeaderSplitParent:
idf = f
case FieldHeaderSplitPrevious:
case FieldHeaderSplitParentSignature:
sigf = f
case FieldHeaderSplitParentHeader:
hdrf = f
break loop
default:
panic("unreachable with num " + strconv.Itoa(int(num)))
}

off = f.To

if off == len(buf) {
break
}
}

return idf, sigf, hdrf, nil
}
Loading
Loading