DGMGRL> DISABLE CONFIGURATION;
SQL> ALTER SYSTEM SET DG_BROKER_START=FALSE;

Perform the database upgrade

SQL> ALTER SYSTEM SET DG_BROKER_START=TRUE;
DGMGRL> ENABLE CONFIGURATION;

The first time 11g broker starts it detects and upgrade the 10g broker files.