XML attachment decoded to MemoryStream doesn't work

My app reads attachments for Gmail. Attachments are kind of XML files (particularly .tcx). When I Decode them to MemoryStream I get an error on

XDocument.Load(stream): 

System.Xml.XmlException
  HResult=0x80131940
  Message=Root element is missing.
  Source=System.Private.Xml
  StackTrace:
   at System.Xml.XmlTextReaderImpl.Throw(Exception e)
   at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
   at System.Xml.XmlTextReaderImpl.Read()
   at System.Xml.Linq.XDocument.Load(XmlReader reader, LoadOptions options)
   at System.Xml.Linq.XDocument.Load(Stream stream, LoadOptions options)
   at System.Xml.Linq.XDocument.Load(Stream stream)

But when I Decode it to FileStream everything is ok.

Failing Code:

foreach (var attachment in message.Attachments)
{
    var stream = new MemoryStream();

    if (attachment is MessagePart rfc822)
    {
        rfc822.Message.WriteTo(stream);
    }
    else
    {
        var part = (MimePart)attachment;
        part.Content.DecodeTo(stream);
    }
    XDocument xDocument = XDocument.Load(stream);
}

But if I use FIleStream, it works:

using (var stream = File.Create(fileName))
{
    if (attachment is MessagePart rfc822)
    {
        rfc822.Message.WriteTo(stream);
    }
    else
    {
        var part = (MimePart)attachment;
        part.Content.DecodeTo(stream);
    }
}

XDocument xDocument = XDocument.Load(File.Open(fileName, 
FileMode.OpenOrCreate));  

You need to rewind the stream back to the beginning before you try to load it.

In other words, do this:

stream.Position = 0;
XDocument xDocument = XDocument.Load(stream);