Mani's Blog

June 22, 2012

Warning: ORA-16607: one or more databases have failed

Filed under: Dataguard,Oracle — mani @ 12:15 am

After the Switchover, you may get a warning “ORA-16607: one or more databases have failed”.   It obviously means you new to restart the new standby database.

1. Display the configuration.

DGMGRL> show configuration verbose
Configuration
  Name:                PRETZEL
  Enabled:             YES
  Protection Mode:     MaxPerformance
  Databases:
    XNOPRTZ   - Primary database
    XNOPRTZDG - Physical standby database
Fast-Start Failover: DISABLED
Current status for "PRETZEL":
Warning: ORA-16607: one or more databases have failed
DGMGRL>
2. Restart the database.
SQL> shutdown immediate;
ORA-01109: database not open
Database dismounted.
ORACLE instance shut down.
SQL> startup mount
ORACLE instance started.
Total System Global Area 2.6724E+10 bytes
Fixed Size          2160272 bytes
Variable Size        1.1543E+10 bytes
Database Buffers     1.5032E+10 bytes
Redo Buffers          146423808 bytes
Database mounted.
SQL>
3. Start the apply process.  Apply command may change based on type of standby.  This is applicable to active Dataguard.
SQL> alter database recover managed standby database cancel;
 
Database altered.
 
SQL> ALTER DATABASE OPEN READ ONLY;
 
Database altered.
 
SQL> alter database recover managed standby database using current logfile disconnect;
 
Database altered.
 
SQL>
4.  Display the config status, make sure the Warning message is gone.
DGMGRL> show configuration verbose
Configuration
  Name:                PRETZEL
  Enabled:             YES
  Protection Mode:     MaxPerformance
  Databases:
    XNOPRTZ   - Primary database
    XNOPRTZDG - Physical standby database
Fast-Start Failover: DISABLED
Current status for "PRETZEL":
SUCCESS
DGMGRL>

June 21, 2012

ORA-16610: command “EDIT DATABASE %S SET PROPERTY” in progress

Filed under: Uncategorized — mani @ 10:43 pm

When I try to set up and configure, ran into Warning message.  I was able to solve the issue as below.

DGMGRL> enable configuration
Enabled.
DGMGRL> show configuration

Configuration
Name:                PRETZEL
Enabled:             YES
Protection Mode:     MaxPerformance
Databases:
XNOPRTZ   – Primary database
XNOPRTZDG – Physical standby database

Fast-Start Failover: DISABLED

Current status for “PRETZEL”:
Warning: ORA-16610: command “EDIT DATABASE XNOPRTZDG SET PROPERTY” in progress

Soultion:  I had to re-start the DG Broker as below.

 

SQL> ALTER SYSTEM SET DG_BROKER_START=FALSE;

System altered.

SQL> ALTER SYSTEM SET DG_BROKER_START=TRUE;

System altered.

SQL>

Setup Oracle Dataguard Broker and Switchover

Filed under: Dataguard,Oracle — mani @ 10:31 pm

Objective

Setting up DG Broker, which can be used to switchover and failover the Database to Physical Standby Database. DG Broker is easier to manage though you could do it from SQL Prompt.

Steps

  1. Check the current state of DG Broker on Primary Database
    SQL> show parameter DG_BROKER_START
    NAME                     TYPE                 VALUE
    ------------------------------------ -------------------------------- ------------------------------
    dg_broker_start              boolean                  FALSE
  2. Start the DG Broker On Primary Database
    SQL> alter system set DG_BROKER_START=true scope=both;
    System altered.
    SQL>
  3. Check the current state of DG Broker on Standby Database
    SQL> show parameter DG_BROKER_START
    NAME                     TYPE                 VALUE
    ------------------------------------ -------------------------------- ------------------------------
    dg_broker_start              boolean                  FALSE
  4. Start the DG Broker On Standby Database
    SQL> alter system set DG_BROKER_START=true scope=both;
    System altered.
    SQL>
  5. Configure Primary Database on Primary DB server.
    [oracle@xnodbprtz01 ~]$ dgmgrl
    DGMGRL for Linux: Version 11.1.0.7.0 - 64bit Production
    Copyright (c) 2000, 2005, Oracle. All rights reserved.
    Welcome to DGMGRL, type "help" for information.
    DGMGRL> connect sys/oracle
    Connected.
    DGMGRL> create configuration 'PRETZEL' as primary database is 'XNOPRTZ' connect identified is xnoprtz;
    create configuration 'PRETZEL' as primary database is 'XNOPRTZ' connect identified is xnoprtz;
                                                                            ^
    Syntax error before or at "identified"
    DGMGRL> create configuration 'PRETZEL' as primary database is 'XNOPRTZ' connect identifier is xnoprtz;
    Configuration "PRETZEL" created with primary database "XNOPRTZ"
    DGMGRL>
  6. Verify the configuration of Primary Database.
    DGMGRL> show configuration;
    Configuration
      Name:                PRETZEL
      Enabled:             NO
      Protection Mode:     MaxPerformance
      Databases:
        XNOPRTZ - Primary database
    Fast-Start Failover: DISABLED
    Current status for "PRETZEL":
    DISABLED
    DGMGRL>
  7. Add Standby DB info on Primary DB Server
    DGMGRL> add database 'XNOPRTZDG' as connect identifier is xnoprtzdg maintained as physical;
    Database "XNOPRTZDG" added
    DGMGRL>
  8. Verify the configuration
    DGMGRL> show configuration
    Configuration
      Name:                PRETZEL
      Enabled:             NO
      Protection Mode:     MaxPerformance
      Databases:
        XNOPRTZ   - Primary database
        XNOPRTZDG - Physical standby database
    Fast-Start Failover: DISABLED
    Current status for "PRETZEL":
    DISABLED
    DGMGRL>
  9. Enable DG Configuration
    DGMGRL> enable configuration
    Enabled.
    DGMGRL> show configuration
    Configuration
      Name:                PRETZEL
      Enabled:             YES
      Protection Mode:     MaxPerformance
      Databases:
        XNOPRTZ   - Primary database
        XNOPRTZDG - Physical standby database
    Fast-Start Failover: DISABLED
    Current status for "PRETZEL":
    Warning: ORA-16610: command "EDIT DATABASE XNOPRTZDG SET PROPERTY" in progress
  10. Restart the DG Broker to get rid of ORA-16610 Warning.
    SQL> ALTER SYSTEM SET DG_BROKER_START=FALSE;
    System altered.
    SQL> ALTER SYSTEM SET DG_BROKER_START=TRUE;
    System altered.
    SQL>
  11. Verify the configuration
    DGMGRL> show configuration
    Configuration
      Name:                PRETZEL
      Enabled:             YES
      Protection Mode:     MaxPerformance
      Databases:
        XNOPRTZ   - Primary database
        XNOPRTZDG - Physical standby database
    Fast-Start Failover: DISABLED
    Current status for "PRETZEL":
    SUCCESS
    DGMGRL>
  12. Get detailed status of the database
    DGMGRL> show database verbose 'XNOPRTZ'
    Database
      Name:            XNOPRTZ
      Role:            PRIMARY
      Enabled:         YES
      Intended State:  TRANSPORT-ON
      Instance(s):
        XNOPRTZ
      Properties:
        DGConnectIdentifier             = 'xnoprtz'
        ObserverConnectIdentifier       = ''
        LogXptMode                      = 'ASYNC'
        DelayMins                       = '0'
        Binding                         = 'OPTIONAL'
        MaxFailure                      = '0'
        MaxConnections                  = '1'
        ReopenSecs                      = '300'
        NetTimeout                      = '30'
        RedoCompression                 = 'DISABLE'
        LogShipping                     = 'ON'
        PreferredApplyInstance          = ''
        ApplyInstanceTimeout            = '0'
        ApplyParallel                   = 'AUTO'
        StandbyFileManagement           = 'AUTO'
        ArchiveLagTarget                = '0'
        LogArchiveMaxProcesses          = '30'
        LogArchiveMinSucceedDest        = '1'
        DbFileNameConvert               = ''
        LogFileNameConvert              = ''
        FastStartFailoverTarget         = ''
        StatusReport                    = '(monitor)'
        InconsistentProperties          = '(monitor)'
        InconsistentLogXptProps         = '(monitor)'
        SendQEntries                    = '(monitor)'
        LogXptStatus                    = '(monitor)'
        RecvQEntries                    = '(monitor)'
        HostName                        = 'xnodbprtz01.ningops.com'
        SidName                         = 'XNOPRTZ'
        StaticConnectIdentifier         = '(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=xnodbprtz01.ningops.com)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=XNOPRTZ_DGMGRL.ningops.com)(INSTANCE_NAME=XNOPRTZ)(SERVER=DEDICATED)))'
        StandbyArchiveLocation          = '/arch/XNOPRTZ'
        AlternateLocation               = ''
        LogArchiveTrace                 = '0'
        LogArchiveFormat                = '%t_%s_%r.arc'
        LatestLog                       = '(monitor)'
        TopWaitEvents                   = '(monitor)'
    Current status for "XNOPRTZ":
    SUCCESS
    DGMGRL>

Switchover to Standby

  1. Switchover to Standby
    DGMGRL> show configuration verbose
    Configuration
      Name:                PRETZEL
      Enabled:             YES
      Protection Mode:     MaxPerformance
      Databases:
        XNOPRTZ   - Primary database
        XNOPRTZDG - Physical standby database
    Fast-Start Failover: DISABLED
    Current status for "PRETZEL":
    SUCCESS
    DGMGRL> switchover to XNOPRTZDG
    Site "xnoprtzdg" was not found
    DGMGRL> switchover to 'XNOPRTZDG'
    Performing switchover NOW, please wait...
    New primary database "XNOPRTZDG" is opening...
    Operation requires shutdown of instance "XNOPRTZ" on database "XNOPRTZ"
    Shutting down instance "XNOPRTZ"...
    ORA-01017: invalid username/password; logon denied
    You are no longer connected to ORACLE
    Please connect again.
    Unable to shut down instance "XNOPRTZ"
    You must shut down instance "XNOPRTZ" manually
    Operation requires startup of instance "XNOPRTZ" on database "XNOPRTZ"
    You must start instance "XNOPRTZ" manually
    Switchover succeeded, new primary is "XNOPRTZDG"
    DGMGRL>
    DGMGRL> show configuration verbose
    Error:
    ORA-16570: database needs restart
    ORA-06512: at "SYS.X$DBMS_DRS", line 228
    ORA-06512: at line 1
    DGMGRL> exit
  2. Original Primary DB need to be restarted as below
    [oracle@xnodbprtz01 ~]$ dba
    SQL*Plus: Release 11.1.0.7.0 - Production on Thu Jun 21 20:11:04 2012
    Copyright (c) 1982, 2008, Oracle.  All rights reserved.
    Connected to:
    Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production
    With the Partitioning, OLAP, Data Mining and Real Application Testing options
    SQL> shutdown immediate;
    ORA-01109: database not open
    Database dismounted.
    ORACLE instance shut down.
    SQL> startup mount
    ORACLE instance started.
    Total System Global Area 2.6724E+10 bytes
    Fixed Size          2160272 bytes
    Variable Size        1.1543E+10 bytes
    Database Buffers     1.5032E+10 bytes
    Redo Buffers          146423808 bytes
    Database mounted.
    SQL>
  3. Start the apply process
    SQL> alter database recover managed standby database cancel;
    Database altered.
    SQL> ALTER DATABASE OPEN READ ONLY;
    Database altered.
    SQL> alter database recover managed standby database using current logfile disconnect;
    Database altered.
    SQL>
  4. Verify Current Status
    DGMGRL> show configuration
    Configuration
      Name:                PRETZEL
      Enabled:             YES
      Protection Mode:     MaxPerformance
      Databases:
        XNOPRTZDG - Primary database
        XNOPRTZ   - Physical standby database
    Fast-Start Failover: DISABLED
    Current status for "PRETZEL":
    SUCCESS
    DGMGRL>

Switch Back to Primary Database

  1. Switch
    DGMGRL> switchover to 'XNOPRTZ'
    Performing switchover NOW, please wait...
    New primary database "XNOPRTZ" is opening...
    Operation requires shutdown of instance "XNOPRTZ" on database "XNOPRTZDG"
    Shutting down instance "XNOPRTZ"...
    ORA-01017: invalid username/password; logon denied
    You are no longer connected to ORACLE
    Please connect again.
    Unable to shut down instance "XNOPRTZ"
    You must shut down instance "XNOPRTZ" manually
    Operation requires startup of instance "XNOPRTZ" on database "XNOPRTZDG"
    You must start instance "XNOPRTZ" manually
    Switchover succeeded, new primary is "XNOPRTZ"
    DGMGRL>
  2. Verify the Status
    DGMGRL> show configuration verbose
    Configuration
      Name:                PRETZEL
      Enabled:             YES
      Protection Mode:     MaxPerformance
      Databases:
        XNOPRTZ   - Primary database
        XNOPRTZDG - Physical standby database
    Fast-Start Failover: DISABLED
    Current status for "PRETZEL":
    Warning: ORA-16607: one or more databases have failed
    DGMGRL>
  3. Restart the database on Standby box
    [oracle@xnodbprtzdg01 dbs]$ dba
    SQL*Plus: Release 11.1.0.7.0 - Production on Thu Jun 21 20:53:57 2012
    Copyright (c) 1982, 2008, Oracle.  All rights reserved.
    Connected to:
    Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production
    With the Partitioning, OLAP, Data Mining and Real Application Testing options
    SQL> shutdown immediate;
    ORA-01109: database not open
    Database dismounted.
    ORACLE instance shut down.
    SQL> startup mount
    ORACLE instance started.
    Total System Global Area 2.6724E+10 bytes
    Fixed Size          2160272 bytes
    Variable Size        1.1543E+10 bytes
    Database Buffers     1.5032E+10 bytes
    Redo Buffers          146423808 bytes
    Database mounted.
    SQL>
  4. Start the apply process
    SQL> alter database recover managed standby database cancel;
     
    Database altered.
     
    SQL> ALTER DATABASE OPEN READ ONLY;
     
    Database altered.
     
    SQL> alter database recover managed standby database using current logfile disconnect;
     
    Database altered.
     
    SQL>
  5. Verify the status finally
    DGMGRL> show configuration verbose
    Configuration
      Name:                PRETZEL
      Enabled:             YES
      Protection Mode:     MaxPerformance
      Databases:
        XNOPRTZ   - Primary database
        XNOPRTZDG - Physical standby database
    Fast-Start Failover: DISABLED
    Current status for "PRETZEL":
    SUCCESS
    DGMGRL>

June 19, 2012

Oracle Agent Installation failed – Completed with Status=255

Filed under: Oracle — mani @ 9:16 pm

Issue:
I was trying to install Oracle Agent 10.2.0.5  on DB server ( It is re-install, I had to remove and install again ).   It failed with “Completed with Status=255”

I spend good amount of time,  Oracle Inventory has some sticky info about the previous installation.   So new inventory location fixed the issue.

Starting Oracle Universal Installer…

Checking installer requirements…

Checking operating system version: must be enterprise-4, enterprise-5, redhat-3, redhat-4, redhat-5, redhat-5.1, SuSE-9, SuSE-10, UnitedLinux-1.0, asianux-1 or asianux-2
Passed

All installer requirements met.

Preparing to launch Oracle Universal Installer from /tmp/OraInstall2012-06-19_06-41-43PM. Please wait …Completed with Status=255
Please specify a Unique Oracle Base directory, or remove previously installed products and their directories. Directory name should contain only valid alphanumeric characters, including ‘_’, or ‘.’ characters.
Removing the copied stuff…..
Removed: /local/home/oracle/agentDownload10.2.0.5.0Oui/oui_linux_x64.jar
Removed: /local/home/oracle/agentDownload10.2.0.5.0Oui/agent_download.rsp
Removed:/local/home/oracle/agentDownload10.2.0.5.0Oui/Disk1

Solution:  Change the Inventory location under /etc/oraInst.loc  and now install the agent.  The issue is fixed.

June 16, 2012

Formula needed for Project Management

Filed under: Project Management — mani @ 12:02 am

Formula
Time Mananagement PERT Expected Activity Duration(EAD) (P+4M+0)/6
Standard Deviation of Activity (P-O)/6
Variance of Activity ((P-O)/6)^2
Range of Activity EAD +/ SD
Total Float LS-ES or LF-EF
Cost Management Schedule Variance EV-PV
Cost Variance EV-AC
Schedule Performance Index(SPI) EV/PV
Cost Performance Index (CPI) EV/AC
Estimate At Completion (EAC) AC + BottomUp ETC
AC + (BAC-EV)
BAC/CPI
AC+((BAC-EV)/(BAC-AC))
To Complete Performance Index (TCPI) (BAC-EV / BAC-AC)
Estimate To Complete (ETC) EAC – AC
Variance At Completion (VAC) BAC-EAC
Communication Management Communication Channel N(N-1)/2
Risk Management Expected Monetary Value (EMV) P X I
Procument Management Point of Total Assumption (PTA) ((Ceiling Price – Target Price) / Buyer’s Share Ratio )) + Target Cost

June 8, 2012

Train your Brain (Brain Exercise)

Filed under: Thought of the day — mani @ 4:30 pm

Train your Brain:

New Brain Cells are continually generated – even in adults – through a process called neurogenesis.  Most new cells are born in hippocampus.

Learning and stress-relieving activities enhance neurogenesis and promote new-cell survival.

Learning new skills creates connections in the brain that counter-act cognitive decline.  It’s never too later to build brainpower with these mental exercises.

Brain Exercises

– Take up violin if you are already playing piano.
– Learn to speak another language.
– If you are crossword puzzle pro, switch to sudoku.
– Use your non-dominant hand to write or brush your teeth.
– Play board games or Computer/Video games that activate strategic, spatial and memory skills.
– Dance.  Intricate choreography fine-tunes the body and brain.

Beyond these exercise

Eat Smart , Exercise regularly and Mind your emotions.

Eat Smart:

If you need a brain-boosting tuneup,  try mediterranean meal plan. Olive, nuts, Avocados, dark leafy green, Vitamin E and Folic Acid.  Colorful fruits and vegetables such as carrots, tomatoes and mangoes.

Exercise:

3 times 40 minutes of aerobic exercise sessions.  Cardiovascular exercise increases blood flow to the brain and decreases your risk of stroke.

Mind your emotions

Build your coping skills.  Adequate sleep, massage, meditation and spiritual practice.

Blog at WordPress.com.