If you experience kind of memory leak, when process is just eating memory without giving OOM or StackOverflow, it would be good to create a dump at the moment when used memory is already high, but not that high to induce an application shutdown or other unwanted consequences.
First of all, you need to download DebugDiag tool: http://www.microsoft.com/en-us/download/details.aspx?id=26798
Then open DebugDiag 1.2 from your Start menu and here are your next steps:
If you experience an excessive memory usage:
If you have a process eating processor time:
Next steps after you choose needed performance counters and their thresholds:
After you start rule, run your faulting application for some time. When user dump is generated, DebugDiag will show "1" or "2" in its main dialog window, under UserDumps column. Then you may just open the folder you set for dumps on the last step and get your nice .dmp file.