Need to download a very large amount of data from a database through a web
service
I need to create an action that downloads certain records from my database
as a text file. The code below works fine for a toy example (few records).
When this code is pushed to production it turns out I run out of memory.
Amount of data I deal with is in order of Gbs
public FileResult Streammer()
{
MemoryStream file = new MemoryStream();
using (var db = new DbMapper())
{
while (true)
{
List<Record> records = db.GetNext(10000);
if (records == null)
break;
foreach (Record r in records)
{
byte[] data = r.GetBytes();
file.Write(data, 0, data.Length);
}
}
}
file.Position = 0;
return new FileStreamResult(file, "text") { FileDownloadName =
"junk.txt" };
}
Ideally, I would like to read chunks of records from the DB on the server.
As a single chunk is read it is immediately converted and streamed to a
client. The client would should a standard download indicator without
actually indicating the remaining time/size.
Any recommendations would be greatly appreciated.
No comments:
Post a Comment