0

Why is Exchange 2013 generating “Transaction logs” for the wrong database while moving mailboxes?

It is a beautiful Friday morning. Sun is shining bright and you are just about to enter your office. You are feeling happy and excited because you finished installing Exchange 2013 two weeks ago on a single physical server; everything is working great. But, you know, you need that second server for high availability so you already made your case to management. After those long conversations where management is complaining about “budget” and you are trying to convince them with “the reasons why you need” it; they finally approved your request. So this morning you received your brand new second physical server.

After few hours, you have your Second Exchange 2013 server and Database Availability Group is up, configured with all the best practices in place.  You do not even realize, how time passed so fast. It is already lunch time! Your buddy joins you at one of your favourite restaurants for a great lunch. And all you talk about is Exchange, Exchange, and Exchange: “What a great day already, now I shall get rid of these databases and create new ones in DAG and move every mailbox there”

You head back to office feeling the passion to finish what you started and you concentrate on planning to move all mailboxes.  You have limited storage until you get rid of those local databases because you planned a new database storage structure after realizing that you can really use Exchange 2013 AutoReseed feature. You carefully calculate the transaction log space for each database. Cross checking mailboxes that you are going to move and you create nice batch files for the new batch migration framework on Exchange 2013 Exchange Administration Center.

You are ready! You have your batch files in CSV format, you are confident that if you move all the mailboxes in those CSV files at once you will not have any problems because you have enough space for database growth and the transaction log space.

You go to Exchange Administration Center. Quickly finding your way to migration page and you create a new migration batch for the first CSV file. You are feeling good and confident but there is something wrong going on. “Why do transaction logs are getting generated on a database which is neither the source or the target! I am going to run out of space on that volume if I run the rest of the batches!!!” You are thinking of turning circular logging on for that database. “But wait, why is this happening? What did I do wrong! What is the problem/solution?

Answer is NOTHING! Nothing is wrong and there is NO solution because this is NOT a problem it is by design…

You just missed an important information that you need to know about moving mailboxes in Exchange 2013. If you are using EAC it will use New-MigrationBatch cmdlet and New-MigrationBatch cmdlet uses a system migration mailbox “Migration.8f3e7716-2011-43e4-96b1-aba62d229136” to keep the metadata.

Microsoft states:

“Assume that you use the Migration wizard in Exchange Administration Center or *-MigrationBatch commands to move mailboxes in Microsoft Exchange Server 2013. In this situation, transaction logs are generated for the mailbox database that hosts the “Migration.8f3e7716-2011-43e4-96b1-aba62d229136″ system mailbox. This issue occurs even though the mailbox database that hosts the Migration mailbox is neither the source or target mailbox database. The transactions logs that are generated may be very large, depending on the size of mailboxes that are moved, and could lead to disk space issues.”

And the cause is

“The behavior is by design. The migration batch framework uses the “Migration.8f3e7716-2011-43e4-96b1-aba62d229136″ arbitration mailbox to store migration metadata (such as batches and migration user information). This behavior generates transaction logs.”

To read the KB Article Click here.

So, while moving mailboxes around databases if you think it will be a problem to generate transaction logs on the database where Migration System Mailbox is (Due to large metadata), than you should consider the following options.

  1. Turn on Circular Logging for the database where Migration System Mailbox is. (Circular logging is evil don’t do it if you have important mailboxes on that database, lol).
  2. Go back to your Exchange 2010 days and use New-MoveRequest to move mailboxes.
  3. Move the transaction log path to a very large volume for the database where Migration System Mailbox is.

Have a nice weekend…

5

Report Exchange 2010 Retention Policy Expiration Information for items on a mailbox (EWS)

Outlook 2010 or Outlook 2013 displays retention information for items on Exchange 2010 but the issue is users usually do not check items which will get expired and causes complaints from time to time. One of the requests was to send a report to user showing items that will expire on certain time period. This is why a few weeks ago I started working on this EWS Script to report each item in a mailbox for retention expiration information.

This script will also help admins to report retention policy tags that applied on items while they are testing Retention Policies.

Description

This script reports Retention Information for all items on a requested mailbox. You can have CSV, HTML or both reports as output, which ever suits you best.

Version Information and Update Information

  • Version 1.0 Released : 18 January 2013

Coming soon…

The following features will be added soon. If you want to request any feature please leave a comment.

  • Email User with the report.
  • Set and Run as Scheduled Job.
  • Add Retention Policy Information to HTML Report

Download

You can download the script from HERE.

Prerequisites

There are some important requirements that needs to be completed before running this script:

  • If you have NOT installed EWS Managed API 2.0 download and install it from to the computer that you will be running this script: http://www.microsoft.com/en-us/download/details.aspx?id=35371
  • You need to import Exchange 2010 snap-in to your PowerShell session or you need to use Exchange Management Shell (Easiest way is to install Management tools and run the script on EMS). If you installed the Exchange 2010 Management tools on your computer you can use: add-pssnapin Microsoft.Exchange.Management.PowerShell.E2010 to add the snap-in to your sessions.
  • This script uses Impersonation. The account you are using for reporting must have impersonation rights on Exchange 2010. For more information on Impersonation:http://msdn.microsoft.com/en-us/library/exchange/dd633680(v=exchg.80).aspx
  • I usually like to use scope for impersonation for security reasons. If you do not care; you can use the following command to give the account impersonation rights that you will be using to run this script:
    • New-ManagementRoleAssignment –Name:impersonationAssignmentName –Role:ApplicationImpersonation –User: YourAccount
  • Exchange 2010 Throttling Policy might cause reporting have missing items. (Even though I am using default values of 1000 for EWSFindCountLimit on the script) If you are going to use this script against large mailboxes make sure Throttling policy is set correctly as it might take longer than expected to get all the information. For more information on Exchange 2010 EWS Throttling please read: http://msdn.microsoft.com/en-us/library/exchange/hh881884(v=exchg.140).aspx. For some cases I have created a policy for this script with the following settings to speed it up:
    • Set-ThrottlingPolicy RetentionPolicyReport -EWSMaxConcurrency $null -EWSPercentTimeInAD $null -EWSPercentTimeInCAS $null -EWSPercentTimeInMailboxRPC $null -EWSMaxSubscriptions $null -EWSFastSearchTimeoutInSeconds $null -EWSFindCountLimit $null -CPUStartPercent $null

*This is not recommended please set the throttling policy according to your needs.

  • This script is using code that Glen Scales wrote for making connections to EWS. Please review lines: 129 to 193 and set the desired connection method. Currently script is going to use current credentials for the logged on user and will use AutoDiscover to determine the EWS address for the mailbox you requested to report. Please read http://gsexdev.blogspot.com/2012/01/ews-managed-api-and-powershell-how-to.html , Glen clearly explains what you need to connect.
  • Script is set to use Exchange 2010_SP2 for exchange version. (Line: 131)

Script Usage and Examples

If all these requirements are set, you are ready to get report on any item on any mailbox (no scoped impersonation required) for Retention information. You can download and save the script to a directory on the computer that you will be running this script. Make sure you have write permission on this directory as the script will be creating the report in the same directory.

Here are some other examples for the script usage:

  • .\Get-RetentionExpiration -Identity Serkan -HTMLReport
    • This will create a HTML Report for Serkan user and will include all mailbox items. HTML Report will be saved in the same folder as the script.
  • .\Get-RetentionExpiration -Identity Serkan -CSVReport
    • This will create a CSV Report for Serkan user and will include all mailbox items. CSV Report will be saved in the same folder as the script.
  • .\Get-RetentionExpiration -Identity Serkan -CSVReport -HideNeverExpires
    • This will create a CSV Report for Serkan user and will include all mailbox items. If item does not have any expiration date it will be excluded. CSV Report will be saved in the same folder as the script.
  • .\Get-RetentionExpiration -Identity Serkan -OnlyTheseFolders “\Inbox*”,”Drafts” -CSVReport
    • This will create a CSV Report for Serkan user and will include Drafts, Inbox and Inbox Subfolders. CSV Report will be saved in the same folder as the script.
  • .\Get-RetentionExpiration -Identity Serkan -ExcludeFolders “\Conversation History*”,”Temp” -CSVReport
    • This will create a CSV Report for Serkan user and will EXCLUDE Temp, Conversation History and Conversation History Subfolders. CSV Report will be saved in the same folder as the script.
  • .\Get-RetentionExpiration -Identity Serkan -ExpiresInDays 60 -CSVReport
    • This will create a CSV Report for Serkan user for items that will expire in next 60 days. CSV Report will be saved in the same folder as the script.
  • .\Get-RetentionExpiration -Identity Serkan -ExpiresInDays 30 -CSVReport -HTMLReport
    • This will create a CSV Report for Serkan user for items that will expire in next 30 days. Both HTML Report and CSV Report will be saved in the same folder as the script.

Here is the screenshot showing the script process:

Get-RetentionExpiration

Here is an example HTML Report, I filtered a certain folder for this example using the –OnlyTheseFolders Parameter.

Folder Name Subject Recieved Date Expiration Date Size Retention Tag
\USER FOLDERS\5 Year Retention Cigdem V, Ticket #1234 01/15/2013 09:12:02 01/14/2018 13:12:02 11161 5 Year
\USER FOLDERS\5 Year Retention CV 12/14/2012 09:30:58 Never Expires 2700 99 Year
\USER FOLDERS\5 Year Retention Had to rename this subject :) 12/12/2012 11:22:17 12/11/2017 15:22:17 2548 5 Year
\USER FOLDERS\5 Year Retention Another Report 11/18/2011 20:37:12 11/17/2016 00:37:12 4775 5 Year
\USER FOLDERS\99 Year Retention Exchange 2010 Retention Script 01/15/2013 11:07:32 Never Expires 4130 99 Year

Note: If the item retention policy tag does not match the folder retention policy tag it will be highlighted on HTML report.

Please let me know if you have any questions or help for this script. I hope it helps.

In addition I would like to thank Glen Scales for his contributions on EWS on his blog.

9

Using Transport Rules to append text to Meeting Requests when Room Mailbox is selected as Resource

This week I got a request to append information to message body of a meeting request when a room mailbox is chosen as a resource. I have to say; at first I was thinking this should be pretty easy with Transport Rules on Exchange 2010 but after several tries this was not true. Before I present the solution I want to walk through the attempts to which I miserably failed to set this rule.

The requested scenario was very simple:

  1. User sends a meeting request to multiple recipients who are on To and CC.
  2. User adds a Room Mailbox which has third party Video Conferencing utility to Resources. (All third party Video Conferencing Utility rooms have 777xxx at the end of display name for easy selection.)
  3. When recipients receive this request we want to see the appended information.

I decided to use “VC Room 777666” Room Mailbox and “INFORMATION ABOUT ROOM MAILBOX” text as information that will be appended for my tests; I created the first Transport Rule for my test:

  • Condition: Sent to “VC Room 777666”
  • Action: Append “INFORMATION ABOUT HOW TO CONNECT”
  • Result: Text only appended to the message received by Room Mailbox none of the other recipients got the information in their meeting Request

So I said, ok let’s try a different approach and added the Room Mailbox to “VC Rooms DG” Distribution Group. (I did not set this group as Room List as Exchange Management Console filtering them out on condition selection.)

  • Condition: Sent to a member of “VC Rooms DG”
  • Action: Append “INFORMATION ABOUT HOW TO CONNECT”
  • Result: Text only appended to the message received by Room Mailbox none of the other recipients got the information in their meeting Request

I was on the wrong path here all these rules were looking for exact condition. But my task was to create a rule that will append the text for all recipients. So finally the concept of how to set the rule was clear in my mind. So came the third try:

  • Condition: When any of the recipients in the To or CC fields is a member of “VC Rooms DG”
  • Action: Append “INFORMATION ABOUT HOW TO CONNECT”

As usual after setting the rule, I sent a simple Email and the text was appended and all recipients got it. But wait! I needed to test Meeting Request to confirm everything is working. I was feeling confident after seeing the Email with appended text that it will work.

Oh noooooo it failed again… How can this be? This rule was able to append the text in Email but not on Meeting Requests? There must be something wrong.

I was stuck. It did not make any sense at all. After spending about half an hour on all these tests I was about to give up. I was thinking this must be a bug or something.

I love problems like these this is the main reason why I am in IT. It was time for troubleshooting to find out why it did not work.

“Pipeline tracing is a diagnostic feature in Microsoft Exchange Server 2010 that enables you to capture diagnostic information about e-mail messages as they encounter transport agents registered on Simple Mail Transfer Protocol (SMTP) events in the transport pipeline”

So guess what, I turned on Pipeline Tracing on my Hub Transport Servers to collect the necessary information and to check if the rule is getting applied.

Set-TransportServer  -identity EXHT  -PipelineTracingSenderAddress SerkanVaroglu –PipelineTracingEnabled $True

I did not make any changes on the latest test rule. I sent another meeting request to same mailboxes with my room mailbox selected as resource for the meeting.

Pipeline Tracing collected all the steps and generated “.eml” files for me. I opened every single one of them and checked the headers.

I found the following entry on the header which surprised me:

X-MS-Exchange-Organization-BCC: VC Room 777666      <VCRoom3@get-mailbox.org>

So as you might guess from the entry when you add Room Mailbox as resource (not in To or CC but as a Resource!) Exchange sets this BCC information on the meeting request.

Clouds are clear and sun is shining again. Now I know why my rule is not working: Room mailbox is not on To or CC field. So, now I have to create a rule for BCC.

So here comes the ultimate rule for my task:

  • Condition: When the ‘X-MS-Exchange-Organization-BCC’ matches ‘777\d\d\d’ (When the message header matches text patterns)
  • Action: Append “INFORMATION ABOUT HOW TO CONNECT”
  • Result:  Life is good….

It worked now when I send an meeting request to any of my room mailboxes which has 777xxx number in the display name it appends the required text to the meeting request for all recipients.

5

Exchange 2010 Datacenter Switchover Troubleshooter Diagram

Last week, the Exchange team released “Exchange 2010 datacenter switchover tool”.

Datacenter Activation Coordination (DAC) mode is a property setting for a database availability group (DAG). DAC mode is disabled by default and should be enabled for all DAGs with two or more members that use continuous replication. DAC mode shouldn’t be enabled for DAGs in third-party replication mode unless specified by the third-party vendor.

If a catastrophic failure occurs that affects the DAG (for example, a complete failure of one of the datacenters), DAC mode is used to control the startup database mount behavior of a DAG. When DAC mode isn’t enabled, and a failure occurs that affects multiple servers in the DAG, when a majority of the DAG members are restored after the failure, the DAG will restart and attempt to mount databases. In a multi-datacenter configuration, this behavior could cause split brain syndrome, a condition that occurs when all networks fail, and DAG members can’t receive heartbeat signals from each other. Split brain syndrome can also occur when network connectivity is severed between the datacenters. Split brain syndrome is prevented by always requiring a majority of the DAG members (and in the case of DAGs with an even number of members, the DAG’s witness server) to be available and interacting for the DAG to be operational. When a majority of the members are communicating, the DAG is said to have quorum.

For example, consider a scenario where the first datacenter contains two DAG members and the witness server, and the second datacenter contains two other DAG members. If the first datacenter loses power and you activate the DAG in the second datacenter (for example, by activating the alternate witness server in the second datacenter), if the first datacenter is restored without network connectivity to the second datacenter, the active databases within the DAG may enter a split brain condition.

This tool is useful if you are using DAC and have not planned your DR scenario yet or you did but still not sure which parameters to use with Stop,Restore and Start-DatabaseAvailabilityGroup commands.

The only downside for me is, this tool is a Powerpoint Presentation so you have to click the buttons in order to proceed without knowing the further steps. That is why I wanted to put it on a diagram with the exact same content.

You can download the Diagram from here.

0

Another Exchange Server Blog :)

I have been blogging in Turkish on www.get-mailbox.org for the last few years about Microsoft Exchange. I think it is time to share a bit more in English as well. From now on I will also be blogging here as much as I can.

I will not be translating or posting what I have written already but I will try to keep both blogs in sync for content. Which I guess will consume a lot of time but I promise to do my best :)