• dfyxA
    link
    fedilink
    arrow-up
    51
    ·
    1 month ago

    Note that this isn’t specific to Go. Reading from stream-like data, be it TCP connections, files or whatever always comes with the risk that not all data is present in the local buffer yet. The vast majority of read operations returns the number of bytes that could be read and you should call them in a loop. Same of write operations actually, if you’re writing to a stream-like object as the write buffers may be smaller than what you’re trying to write.

    • bl_r@lemmy.dbzer0.com
      link
      fedilink
      arrow-up
      12
      ·
      1 month ago

      I’ve run into the same problem with an API server I wrote in rust. I noticed this bug 5 minutes before a demo and panicked, but fixed it with a 1 second sleep. Eventually, I implemented a more permanent fix by changing the simplistic io calls to ones better designed for streams

      • dfyxA
        link
        fedilink
        arrow-up
        7
        ·
        1 month ago

        The actual recommended solution is to just read in a loop until you have everything.

    • xthexder@l.sw0.com
      link
      fedilink
      arrow-up
      3
      ·
      1 month ago

      Ah yes… several years ago now I was working on a tool called Toxiproxy that (among other things) could slice up the stream chunks into many random small pieces before forwarding them along. It turned out to be very useful for testing applications for this kind of bug.