c++ - boost::asio fails to read more than 65536 bytes from file -


i'm failing read more 65536 bytes buffer file using boost::asio::windows::stream_handle asynchronously.

starting 65537th byte buffer contains the data beginning of file, rather expected data.

here code example, reproduces issue:

auto handle = ::createfile(l"bigfile.xml", generic_read, file_share_read, nullptr, open_existing, file_flag_overlapped, nullptr); boost::asio::io_service ios;  boost::asio::windows::stream_handle streamhandle(ios, handle);  const auto to_read_bytes = 100000; char buffer[to_read_bytes];  boost::asio::async_read(streamhandle, boost::asio::buffer(buffer, to_read_bytes), [](auto &ec, auto read) {     std::cout << "bytes read: " << read << std::endl; });  ios.run();  auto bufferbegin = std::string(buffer, 38); auto buffercorrupted = std::string(buffer + 65536, 38);   // <- contains bytes beginning of file  std::cout << "offset 0: " << bufferbegin << std::endl; std::cout << "offset 65536: " << buffercorrupted << std::endl;     ::closehandle(handle); 

that code produces output:

> bytes read: 100000   > offset 0: <?xml version="1.0" encoding="utf-8"?>   > offset 65536: <?xml version="1.0" encoding="utf-8"?> 

the source file bigger 65536.

this reproducible boost 1.61 + vs2015. issue in boost 1.55 + vs2010.
operating systems are: windows 7 , windows server 2008r2.

my questions are:
1. known limitation in boost::asio or in winapi?
2. if known limitation, safe size of buffer read data? safe have buffer of size 65536, or should smaller?

this neither limitation of asio, windows, nor buffer sizes. rather, asio performing has been told within specifications: reading 100000 bytes regular file as-if stream. windows::stream_handle:

  • async_read() composed of 0 or more intermediate async_read_some() operations until either number of bytes requested application has been transferred, or until error occurs

    this operation implemented in terms of 0 or more calls stream's async_read_some function, , known composed operation.

  • async_read_some() operations may read less number of requested bytes

    the read operation may not read of requested number of bytes.

  • each intermediate async_read_some() operation read start of stream

as file handle being used not stream, rather regular file, consider using windows::random_access_handle , async_read_at(device, 0, ...). random-access handles documentation notes:

boost.asio provides windows-specific classes permit asynchronous read , write operations performed on handles refer regular files.

when using windows::random_access_handle , async_read_at():

  • async_read_at() composed of 0 or more intermediate async_read_some_at() operations until either number of bytes requested application has been transferred, or until error occurs
  • async_read_some_at() operations may read less number of requested bytes
  • each intermediate async_read_some_at() operation use offset corresponding end of previous read when reading device (e.g. initial offset + current bytes transferred)

Comments