c# - OpenXML hanging while writing elements -

i have program pulls data database, caches file , exports data multiple formats (excel, excel 2003, csv). i'm using openxml sdk 2.0 excel work. these export processes run in parallel (using parallel.foreach), , amount of data can pretty large - e.g. csvs 800mb. during these larger exports, i've noticed writing of xml documents hang. instance, if have 8 exporting in parallel, @ point "pause". hang around same point:

//this.writer openxmlwriter created worksheetpart. this.writer.writeelement(new cell() {     cellvalue = new cellvalue(value),     datatype = cellvalues.string }); 

when happens, pause debugger (vs2013 in case) , notice threads blocking around same portion of code - bit deeper in openxml sdk - stem call openxmlwriter.writeelement.

i dug through source using justdecompile didn't find answers. appears there intermediary stream in use writing isolated storage , is, reason, blocking. underlying stream each of these filestream.

here screenshot showing (8 in case) parallel tasks blocked @ or inside openxmlwriter.writeelement method:

complete stack 1 of these hung threads - annotations.

windowsbase.dll!ms.internal.io.packaging.packagingutilities.createuserscopedisolatedstoragefilestreamwithrandomname normal windowsbase.dll!ms.internal.io.packaging.packagingutilities.createuserscopedisolatedstoragefilestreamwithrandomname(int retrycount, out string filename)      windowsbase.dll!ms.internal.io.packaging.sparsememorystream.ensureisolatedstorestream()    //---> why writing isolated storage @ all? windowsbase.dll!ms.internal.io.packaging.sparsememorystream.switchmodeifnecessary()   windowsbase.dll!ms.internal.io.zip.zipiofileitemstream.write(byte[] buffer, int offset, int count)    system.dll!system.io.compression.deflatestream.writedeflateroutput(bool isasync)      system.dll!system.io.compression.deflatestream.write(byte[] array, int offset, int count)     windowsbase.dll!ms.internal.io.packaging.compressstream.write(byte[] buffer, int offset, int count)   windowsbase.dll!ms.internal.io.zip.progressivecrccalculatingstream.write(byte[] buffer, int offset, int count)    windowsbase.dll!ms.internal.io.zip.zipiomodeenforcingstream.write(byte[] buffer, int offset, int count)   system.xml.dll!system.xml.xmlutf8rawtextwriter.flushbuffer()      system.xml.dll!system.xml.xmlutf8rawtextwriter.writeattributetextblock(char* psrc, char* psrcend)     system.xml.dll!system.xml.xmlutf8rawtextwriter.writestring(string text)   system.xml.dll!system.xml.xmlwellformedwriter.writestring(string text)    documentformat.openxml.dll!documentformat.openxml.openxmlelement.writeattributesto(system.xml.xmlwriter xmlwriter)    documentformat.openxml.dll!documentformat.openxml.openxmlelement.writeto(system.xml.xmlwriter xmlwriter)      documentformat.openxml.dll!documentformat.openxml.openxmlpartwriter.writeelement(documentformat.openxml.openxmlelement elementobject)     //---> @ point, threads seem blocking.  myproject.common.dll!myproject.common.export.excelwriter.writeline(string[] values) line 117 

one more thing worth mentioning while there 8 things (in case) being exported @ once, each individual exporter writing many files in series. instance, given export may have 150 underlying files exporting - input data segmented , portion written each file. basically, cache bulk data database, read line , push (in series - one-by-one) streams should include data. point if there 8 exporters running, there be, maybe, 1,000 files being written 8 actively writing @ given time.

i know question pretty old, known microsoft issue openxml-isolatedfilestorage. can read workaround here http://support.microsoft.com/kb/951731:

the isolatedstoragefile class not thread safe, isolatedstoragefile static , shared between packagepart objects. when multiple packagepart streams using isolatedstoragefile objects buffer data accessed writing (includes flushing well), thread safety problem in isolatedstoragefile class exposed, causing deadlock.

the basic idea wrap stream of packagepart , lock writing it. pointed example wrapped stream. here implementation:

public class packagepartstream : stream {     private readonly stream _stream;      private static readonly mutex mutex = new mutex(false);      public packagepartstream(stream stream)     {         _stream = stream;     }      public override long seek(long offset, seekorigin origin)     {         return _stream.seek(offset, origin);     }      public override void setlength(long value)     {         _stream.setlength(value);     }      public override int read(byte[] buffer, int offset, int count)     {         return _stream.read(buffer, offset, count);     }      public override void write(byte[] buffer, int offset, int count)     {         mutex.waitone(timeout.infinite, false);         _stream.write(buffer, offset, count);         mutex.releasemutex();     }      public override bool canread     {         { return _stream.canread; }     }      public override bool canseek     {         { return _stream.canseek; }     }      public override bool canwrite     {         { return _stream.canwrite; }     }      public override long length     {         { return _stream.length; }     }      public override long position     {         { return _stream.position; }         set { _stream.position = value; }     }      public override void flush()     {         mutex.waitone(timeout.infinite, false);         _stream.flush();         mutex.releasemutex();     }      public override void close()     {         _stream.close();     }      protected override void dispose(bool disposing)     {         _stream.dispose();     } } 

and example of usage:

var worksheetpart = document.workbookpart.addnewpart<worksheetpart>(); var worksheetwriter = openxmlwriter.create(new packagepartstream(worksheetpart.getstream())); worksheetwriter.writestartelement(new worksheet()); //rest of code goes here ...