Grpc error while marshaling proto marshal called with nil

«proto: Marshal called with nil» after 1.34.0 upgrade #4094

What version of gRPC are you using?

What did you do?

After updating to 1.34.0 we started to get proto: Marshal called with nil when gRPC method is called and nil is passed as request parameter. I believe it is because of this change. I agree, it is wrong, and I am currently fixing our code base but this is sort of breaking change and needs to be called out in release notes at least.

The text was updated successfully, but these errors were encountered:

Ideally we would like to fix this. But looking at the code, it is not very clear to us how it did not panic earlier. We would like to understand better. Were you passing a nil value which implemented the proto.Marshaler interface?

No, it was just pure nil but when nil is assigned to typed param which then passed to interface<> param it becomes interface with type and nil value. Similar to: https://play.golang.org/p/sDJBWQweAYw

Actually I found only case in our code and it is fixed here: temporalio/temporal#1067 ( wfClient is gRPC generated client. Btw, we use gogo, but I think it doesn’t matter as it calls standart grpc lib anyway).

This issue is labeled as requiring an update from the reporter, and no update has been received after 6 days. If no update is provided in the next 7 days, this issue will be automatically closed.

I did my best. Please let me know if I can provide more info.

Btw, we use gogo, but I think it doesn’t matter as it calls standart grpc lib anyway

With the standard protobuf library, we always got panics when passing nil messages:

Seems like this was only a change for gogoproto users using nil messages. As mentioned in https://github.com/grpc/grpc-go/pull/3958/files#r571175706:

gogoproto is not officially supported by gRPC-Go.

If you want a codec that suits your needs better, it’s easy to install your own. Just copy/paste this code into another package, update the parts that are important, and import it in your main package after any grpc imports. (If you don’t control main , you can give it a new name and use the Codec Call -/ Dial -/ Server — Option s.)

Источник

Compilation error: grpc: error while marshaling: proto: Marshal called with nil #1812

Describe the problem

Sorry there is probably something screwy in this code, that I am hacking up.

I tried to do a verify operation, and the build aborted with an error:

That was all of the output, all in orangish color.

I have no idea what that error is supposed to imply.

To reproduce

Not sure, but I included the code. Note: this code is not done yet, nor may never be used, but just an experiment on Teensy 3.2, to see if I can use FTM timer to capture pulse widths and maybe generate specific pulses.

I simply clicked on verify with this sketch, which includes a header file, which is part of the sketch.

Expected behavior

I am guessing probably lots of error messages.

Arduino IDE version

Operating system

Operating system version

Additional context

I am running with yesterdays IDE did not see any updates for today.

Issue checklist

• I searched for previous reports in the issue tracker
• I verified the problem still occurs when using the latest nightly build
• My report contains all necessary details

The text was updated successfully, but these errors were encountered:

I believe I figured it out what happened.

Before this I had done a SaveAs operation to this new name. The code hung with spinning . So I closed out the window (and process). Restarted IDE. Browsed to the new saved file and tried to do the compile.

But turned out it had forgotten I was building for T3.1/2 and had the simple chose board. in drop down list.
Going back up and choose T3.1 now it behaves normal.

Side note: Thought maybe had issue with the new sketch name. So I exited the IDE, used browser to rename the directory and sketch to different name. I then restarted the IDE. Looks like it had issue with not finding the last sketch that was last edited, and gave me a blank window. Also the Sketchbook tab area showed as empty, but was able to go up to File ->Sketchbook and open up the sketch with the new name.

had the simple chose board. in drop down list.

Do you mean the » ❌ No board selected» like this?

I gave it a try but I only got the expected «Missing FQBN» message rather than the unexpected error you reported.

I exited the IDE, used browser to rename the directory and sketch to different name. I then restarted the IDE. Looks like it had issue with not finding the last sketch that was last edited, and gave me a blank window.

I guess it is the same issue you reported at arduino/arduino-ide#1089. That will be fixed by arduino/arduino-ide#1152

Источник

Gorums cannot handle nil for non-PerNodeArg Gorums RPC calls. #35

For my read-write distributed storage implemented by Gorums,
if I send a nil message through either read or write quorum call,
then the quorum call returns a nil reply with error nil.

The reason for this issue is the callGRPC methods generated by Gorums.

For example in method: callGRPCWrite:

This allows the write quorum call to send a nil write request and reply a nil as the write response with error message nil. (the quorum function can receive enough replies to obtain a quorum)

So, in my quorum functions, I currently wrote some code to handle the nil replies, otherwise, it can cause panic, since there’s no value and timestamp to get if replies are nils.

The text was updated successfully, but these errors were encountered:

I’m not sure I understand the issue here.
gRPC doesn’t allow passing nil, as the marshalling step will panic (as it should).
It seems that we now protect against passing nil values on all methods generated by Gorums ( #12).

I think that passing nil values should panic, as this is the developer’s mistake.
There is however one special case where I think we should handle nil values: when a per-node-arg function returns nil, we should drop that request and reduce the expected number of replies for that call. This would allow us to do some cool things with the per-node-arg functions.
What do you think @meling?

The quorum call actually can invoke gRPC calls with nil message and the quorum functions can also receive enough replies (also nil) to obtain a quorum.

So, the returned reply for this quorum call in this case is nil, however if the server failure happens, then no quorum is obtain, the returned reply is also nil. Therefore, I think in my tests, this two cases return the same values (both nils) (one is not an error (the err is nil) according to Hein, another is caused by the server failure), so I can not check the difference.

I think Hein has a solution about this issue. 😉

A quorum call works with nil values as we circumvent the actual gRPC call:

An actual gRPC call would panic with:

@meling: If you look at config_qc_test.go#L928, a request is ignored but the nil values is still added to the reply slice, i.e., we are allowing a write request which needs a quorum of at least 3 with only 2 actual responses. The call succeeds as len(replies) is > 2 (as the nil values is counted).

See PR #36 for a potential fix.

It seems my quick and dirty initial solution was a bit too simplistic; I was hoping to avoid the extra noise around the <> conditional in the template. I’ve made some more changes to the PR, but there is one outstanding issue that needs to be discussed. It turns out that providing a nil argument does not panic in all cases. Take a look at the test in config_qc_test.go#L140. This does not panic when its argument is nil, whereas a similar test with the WritePerNode method does panic; see config_qc_test.go#L956. This is presumably because the Read method takes a ReadRequest , which is actually never used on the server-side. So it seems that it is allowed by gRPC to invoke with a nil argument, e.g., when the server side does not need anything from the client. Will think some more about this. Thoughts anyone?

Added another commit to the fix-nil-in-replies branch. Turns out that it is not easy to check for panic from the Write call since the panic actually happens in another goroutine separate from the calling goroutine running the tests, and so the standard recover() call does not work. One solution could be to check for nil argument to the quorum call and panic immediately, but then also the Read method will fail, even though it works now.

The routine that caused the panic needs to invoke recover, or we might never learn about the panic (expect if we sync with the routine at some point).
Maybe we can pass an err chan error to the function invoked as a go routine, that must be read before checking the actual response value?

My initial thoughts on this is that it seems to be a bit much hassle for handling nil which nobody really should be passing anyway, even though it works for the Read call. I’m continuing to ponder this.

Yes, that’s what I’m thinking as well, we should not try to code ourselves around developer error, let it panic. Though, we should recover a crashing go routine and communicate that back to the calling quorum call, so that the panic is actually caught. I.e., the panic needs to be invoked on the main thread so that the program actually panics.

Don’t you think it is a bit of an overkill to complicate things by recovering individual goroutines in order to pass that on to the main goroutine. Not sure if that helps debug the actual problem, especially if the panic is caused by another issue that we haven’t thought about yet. I think I prefer to let it panic as it does now. Though we cannot write a test for nil and recover from it.

Источник

Develop gRPC API 10x faster

API Definition

You can skip this step and use the generated SDK.

The whole app will consist of only one service — the Book Service.

Next, we will generate the SDK using the GenDocu Console. We will use our protobuf definition available at https://github.com/gendocu-com-examples/library-app/ in proto directory. Create a new project in GenDocu Console, and then fill in the fields as in the image.

Click “create project”. You should be redirected to the project board. After a couple of seconds, the API definition is processed.

When the build is complete, you can click Public docs and then use the generated SDK.

Develop gRPC API 10x faster

• 1 minute to setup
• Direct integration with GitHub
• Autogenerate gRPC and gRPC-Web SDK
• Live-interact with gRPC endpoints

Backend implementation

You can skip this step and use our gRPC Web backend https://t30z1m0w81.execute-api.us-east-1.amazonaws.com .

Currently, GenDocu doesn’t generate the server-side code for gRPC, but we can easily write it on our own. To simplify our code, we have used the GenDocu’s generated SDK — you can fetch it from the Golang client code snippet. Create your Golang project and initialize it with go mod init command. Let’s create a simple HTTP server:

We can run it using the command go run main.go .

The next step is to add the gRPC Web server. We need to install the SDK generated by GenDocu go get -u git.gendocu.com/gendocu/LibraryApp.git/sdk/go/. . After that, we can use the SDK:

The last step is to implement the BookService and register it in the SDK. The good thing is that all the missing methods cause the compilation error.

Then we can register it in the main method:

We have the whole gRPC Web server. We only need to add the CORS headers, and we can move on to the frontend app.

Frontend implementation

You can skip this part and simply use the ready frontend app from our GitHub repository.

We can start from the snippet generated by GenDocu documentation. It contains the predefined remote URL address by default — let’s replace it with the local server localhost:8003 . We will bootstrap our solution from the React template npx create-react-app my-app —template typescript . The next step is to replace the App.tsx file.

To make it work, we need to install the generated Typescript SDK:

After that, we need to add only the gRPC dependencies to allow us to use the protobuf types

Voilà! We should be able to React app that communicates using gRPC Web. Run npm start to start the app.

Let’s add some form to add some books.

We also need to implement the book list functionality.

Now we can list all books, right after adding a new one:

You should be able to add a new book to our list. We can add “1984” with ISBN “9780151660346” by “George Orwell”.

Last but not least — deleting the book from the directory.

Now, after clicking on the book title, we will remove it. It is not the best UX decision but should be fine for our example. Congratulations! You have now completely working local gRPC Web app. There is only one thing that we need to do to make it work with AWS Lambda backend — we need to obtain the backend address from the env variable:

The complete code of the frontend is here. Now, let’s move on to deploying the backend to AWS lambda.

Deployment to AWS — Serverless framework

Before deploying our backend to AWS Lambda, we need to adjust the code. Instead of raw HTTP requests, we need to handle the events. We will convert those events to HTTP requests using the aws-lambda-go-api-proxy . That’s the new main function implementation:

Deployment to AWS might be kind of tricky — gRPC Web requests contain a binary payload. We need to use the API Gateway HTTP API. To properly set it up, we will use the Serverless framework. Please install it using command npm install serverless@2.35.0 . Before proceeding you need to set up the serverless framework — you can find more here. You can also implement those changes manually using the AWS Console.

This our deployment definition:

Next, run the command serverless deploy . After few minutes, you should obtain the lambda endpoint.

Run the react app with the new endpoint: REACT_APP_BACKEND=https://t30z1m0w81.execute-api.us-east-1.amazonaws.com yarn start .

One problem with our app is that we store the book list in the memory in single lambda, and the received book list might be invalid.

Bonus: adding the database for persisting booklist

Our AWS Lambda doesn’t persist the book list — we store it in the memory, so we lose it when lambda is down. To resolve that issue, we will add the DynamoDB table using the serverless framework (or similar Cloudformation template):

and a permission for our lambda to read the table:

We need to adjust our backend implementation. Let’s implement the new version of our Book Service:

With that simple change, we’re storing our booklist in the DynamoDB Database. That’s all!

Troubleshooting

Backend

1. grpc: server failed to encode response: rpc error: code = Internal desc = grpc: error while marshaling: proto: Marshal called with nil . You are possibly returning the nil instead of empty.Empty struct. e.g. replace

gRPC requires HTTP/2 . AWS API Gateway is handling HTTP 2.0 in a tricky way. All options requests are passed to lambda as HTTP 1.1 — gRPC requires using HTTP 2.0 to properly handle that. We have simply early return on options request. POST requests are passed properly as HTTP 2.0.

Error: Response closed without grpc-status (Headers only) . When there is an error, the gRPC Web doesn’t return a body but sets a specific message and code in the headers. Please check whether your function returns the headers (in AWS Lambda settings, in the test section) — grpc-status should be set and be returned in AWS event as headers (not multiValueHeaders ), e.g.

For Golang, there was an issue with the github.com/awslabs/aws-lambda-go-api-proxy that converts http response to API Gateway event — since version 0.10.1 it works fine.

Frontend

Error: Response closed without headers . There are multiple reasons for that error message. Please check in the browser console the request:

• if you can’t find it request log you have possibly misspelled the backend url. Please remember, that you need to provide the whole url including the protocol scheme. e.g. http://localhost:8003 instead of localhost:8003
• check the protocol scheme — it is important to correctly distinguish the http endpoint from https endpoint
• check the response body — you may receive the malformed output data, e.g. json instead of gRPC Web or double base64 encoded data (it is quite common for AWS API Gateway v1)
• check the whole service url — you may misspell it, like http://localhost:8030 instead of http://localhost:8003 . There should be no trailing slash — http://localhost:8003/ is invalid.

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://localhost:8003/gendocu.sdk_generator.v3.BookService/ListBooks. (Reason: CORS request did not succeed). . You need to CORS headers, like:

We strongly advise to not set those headers’ values to ‘*’. You can read more about the correct values here.

Источник

status, err := cli.GetStatus(ctx, &empty.Empty{})
if err != nil {
    return err
}

if status == nil {
    // this should NEVER happen - right?
    return fmt.Errorf("nil Status result returned") 
}

status, err := cli.GetStatus(ctx, nil) // <- runtime error

if err != nil {
    // "rpc error: code = Internal desc = grpc: error while marshaling: proto: Marshal called with nil"
    return err
}

2 ответа

Лучший ответ

func (s *mygRPC) GetStatus(context.Context, *empty.Empty) (*pb.Status, error) {
    log.Println("cli: GetStatus()")

    //return &pb.Status{}, nil
    return nil, nil // <- can server return a nil status message (with nil error)
}

ERROR: rpc error: code = Internal desc = grpc: error while marshaling: proto: Marshal called with nil

2019/05/14 16:09:50 cli: GetStatus()
ERROR: 2019/05/14 16:09:50 grpc: server failed to encode response:  rpc error: code = Internal desc = grpc: error while marshaling: proto: Marshal called with nil

func (s *Server) sendResponse(t transport.ServerTransport, stream *transport.Stream, msg interface{}, cp Compressor, opts *transport.Options, comp encoding.Compressor) error {
    data, err := encode(s.getCodec(stream.ContentSubtype()), msg)
    if err != nil {
        grpclog.Errorln("grpc: server failed to encode response: ", err)
        return err
    }
    // ...
}

func encode(c baseCodec, msg interface{}) ([]byte, error) {
    if msg == nil { // NOTE: typed nils will not be caught by this check
        return nil, nil
    }
    b, err := c.Marshal(msg)
    if err != nil {
        return nil, status.Errorf(codes.Internal, "grpc: error while marshaling: %v", err.Error())
    }
    // ...
}

if msg == nil { // NOTE: typed nils will not be caught by this check }

Investigating further with a contrived server example:

func (s *mygRPC) GetStatus(context.Context, *empty.Empty) (*pb.Status, error) {
    log.Println("cli: GetStatus()")

    //return &pb.Status{}, nil
    return nil, nil // <- can server return a nil status message (with nil error)
}

and testing client/server reactions:

CLIENT:

ERROR: rpc error: code = Internal desc = grpc: error while marshaling: proto: Marshal called with nil

SERVER:

2019/05/14 16:09:50 cli: GetStatus()
ERROR: 2019/05/14 16:09:50 grpc: server failed to encode response:  rpc error: code = Internal desc = grpc: error while marshaling: proto: Marshal called with nil

So even if one wanted to legitimately return a nil value, the gRPC transport will not allow it.

Note: the server-side code is still executed — as expected — but as far as the client is concerned, the gRPC call failed.

Conclusion: a valid (err==nil) server response will always return a valid (non-nil) message.

EDIT:

Inspecting the gRPC source reveals where a nil message is caught:

server.go

func (s *Server) sendResponse(t transport.ServerTransport, stream *transport.Stream, msg interface{}, cp Compressor, opts *transport.Options, comp encoding.Compressor) error {
    data, err := encode(s.getCodec(stream.ContentSubtype()), msg)
    if err != nil {
        grpclog.Errorln("grpc: server failed to encode response: ", err)
        return err
    }
    // ...
}

rpc_util.go

func encode(c baseCodec, msg interface{}) ([]byte, error) {
    if msg == nil { // NOTE: typed nils will not be caught by this check
        return nil, nil
    }
    b, err := c.Marshal(msg)
    if err != nil {
        return nil, status.Errorf(codes.Internal, "grpc: error while marshaling: %v", err.Error())
    }
    // ...
}

The comment in this line is key:

if msg == nil { // NOTE: typed nils will not be caught by this check }

So if one were to use reflect on our typed-nil, reflect.ValueOf(msg).IsNil() would return true. The following c.Marshal(msg) errors — and the call fails to send a message response to the client.