http://trust.salesforce.com Real-time updates from trust.salesforce.com on alerts, availability and system health Fri, 17 May 2013 00:00:00 GMT 2013-05-17T00:00:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team continues to closely monitor performance on the NA6 instance. The system has been stable since performing the emergency maintenance and we continue to investigate root cause. Once the root cause analysis is complete, we will post it to www.trust.salesforce.com. Fri, 17 May 2013 00:00:00 GMT http://trust.salesforce.com/trust/status 2013-05-17T00:00:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team continues to closely monitor performance on the NA6 instance. The system has been stable since performing the emergency maintenance and we continue to investigate root cause. Once the root cause analysis is complete, we will post it to www.trust.salesforce.com. Thu, 16 May 2013 00:00:00 GMT http://trust.salesforce.com/trust/status 2013-05-16T00:00:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team continues to closely monitor performance on the NA6 instance. The system has been stable since performing the emergency maintenance and we continue to investigate root cause. Once the root cause analysis is complete, we will post it to www.trust.salesforce.com. Wed, 15 May 2013 07:30:00 GMT http://trust.salesforce.com/trust/status 2013-05-15T07:30:00Z http://trust.salesforce.com/trust/status The salesforce.com emergency maintenance is now successfully complete. Thank you for your patience during the maintenance window. Wed, 15 May 2013 02:50:00 GMT http://trust.salesforce.com/trust/status 2013-05-15T02:50:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology team is continuing with emergency maintenance to address the recent issues on NA6. Service is now generally available however customers may still experience slowness or intermittent errors accessing the application while the team continue to work to restore full service. Thank you for your continued business and patience during this situation. Wed, 15 May 2013 02:30:00 GMT http://trust.salesforce.com/trust/status 2013-05-15T02:30:00Z http://trust.salesforce.com/trust/status The NA6 salesforce.com scheduled maintenance is beginning. During this time, the salesforce.com service will not be available. Users attempting to access the service during this time will be presented with a scheduled maintenance notification page. We appreciate your patience during this maintenance window. Wed, 15 May 2013 01:55:00 GMT http://trust.salesforce.com/trust/status 2013-05-15T01:55:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology team will conduct an emergency maintenance on the NA6 instance on May 14 at 7:00 PM PDT. The maintenance window is targeted to last 1 hour. The goal of this maintenance is to continue to troubleshoot the recent performance degradations on the NA6 instance. Our top teams and our vendors continue to work together to identify the root cause and resolve it as quickly as possible. We appreciate your patience at this time. Wed, 15 May 2013 00:00:00 GMT http://trust.salesforce.com/trust/status 2013-05-15T00:00:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology team will conduct an emergency maintenance on the NA6 instance on May 14 at 7:00 PM PDT. The maintenance window is targeted to last 1 hour. The goal of this maintenance is to continue to troubleshoot the recent performance degradations on the NA6 instance. Our top teams and our vendors continue to work together to identify the root cause and resolve it as quickly as possible. We appreciate your patience at this time. Tue, 14 May 2013 22:05:00 GMT http://trust.salesforce.com/trust/status 2013-05-14T22:05:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team continues to investigate intermittent performance degradation issues on the NA6 instance. We continue to work with our vendors to execute a plan to fix this issue. Until we fully resolve the issue, customers may experience intermittent periods of slow performance while using the Salesforce application. Please check the status of trust.salesforce.com frequently for updates regarding this issue. Thank you for your continued patience during this situation. Tue, 14 May 2013 21:00:00 GMT http://trust.salesforce.com/trust/status 2013-05-14T21:00:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team continues to investigate intermittent performance degradation issues on the NA6 instance. We continue to work with our vendors to execute a plan to fix this issue. Until we fully resolve the issue, customers may experience intermittent periods of slow performance while using the Salesforce application. Please check the status of trust.salesforce.com frequently for updates regarding this issue. Thank you for your continued patience during this situation. Tue, 14 May 2013 19:00:00 GMT http://trust.salesforce.com/trust/status 2013-05-14T19:00:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team continues to investigate intermittent performance degradation issues on the NA6 instance. We continue to work with our vendors to execute a plan to fix this issue. Until we fully resolve the issue, customers may experience intermittent periods of slow performance while using the Salesforce application. Please check the status of trust.salesforce.com frequently for updates regarding this issue. Thank you for your continued patience during this situation. Tue, 14 May 2013 16:58:00 GMT http://trust.salesforce.com/trust/status 2013-05-14T16:58:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team is working to isolate a performance degradation issue on the NA6 instance. Please check the status of trust.salesforce.com frequently for updates regarding this issue. Tue, 14 May 2013 14:10:00 GMT http://trust.salesforce.com/trust/status 2013-05-14T14:10:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team continues to investigate some intermittent performance issues that have affected the NA6 instance. The system is currently stable, but we continue to monitor performance and investigate root cause. Once the root cause analysis is complete, we will post it to www.trust.salesforce.com. Tue, 14 May 2013 02:56:00 GMT http://trust.salesforce.com/trust/status 2013-05-14T02:56:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team continues to work on isolating the performance degradation issue on the NA6 instance. Customers may continue to see intermittent slowness while accessing the Salesforce application. Please check the status of trust.salesforce.com frequently for updates regarding this issue. Tue, 14 May 2013 00:08:00 GMT http://trust.salesforce.com/trust/status 2013-05-14T00:08:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team continues to work on isolating the performance degradation issue on the NA6 instance. Customers may continue to see intermittent slowness while accessing the Salesforce application. Please check the status of trust.salesforce.com frequently for updates regarding this issue. Mon, 13 May 2013 21:30:00 GMT http://trust.salesforce.com/trust/status 2013-05-13T21:30:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team continues to work on isolating the performance degradation issue on the NA6 instance. Customers may continue to see intermittent slowness while accessing the Salesforce application. Please check the status of trust.salesforce.com frequently for updates regarding this issue. Mon, 13 May 2013 20:30:00 GMT http://trust.salesforce.com/trust/status 2013-05-13T20:30:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team continues to work on the performance degradation issue on the NA6 instance. At this time customers may begin to observe performance return to normal. We will continue to monitor and investigate. Thank you for your continued patience. Please check the status of trust.salesforce.com frequently for updates regarding this issue. Mon, 13 May 2013 19:29:00 GMT http://trust.salesforce.com/trust/status 2013-05-13T19:29:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team continues to work on isolating the performance degradation issue on the NA6 instance. Customers may continue to see intermittent slowness while accessing the Salesforce application. Please check the status of trust.salesforce.com frequently for updates regarding this issue. Mon, 13 May 2013 18:50:00 GMT http://trust.salesforce.com/trust/status 2013-05-13T18:50:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team continues to work on isolating the performance degradation issue on the NA6 instance. Customers may continue to see intermittent slowness while accessing the Salesforce application. Please check the status of trust.salesforce.com frequently for updates regarding this issue. Mon, 13 May 2013 17:50:00 GMT http://trust.salesforce.com/trust/status 2013-05-13T17:50:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team continues to work on isolating the performance degradation issue on the NA6 instance. Customers may continue to see slowness or intermittent errors while accessing the Salesforce application at this time. Please check the status of trust.salesforce.com frequently for updates regarding this issue. Mon, 13 May 2013 17:10:00 GMT http://trust.salesforce.com/trust/status 2013-05-13T17:10:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team is working to isolate a performance degradation issue on the NA6 instance. Please check the status of trust.salesforce.com frequently for updates regarding this issue. Mon, 13 May 2013 16:31:00 GMT http://trust.salesforce.com/trust/status 2013-05-13T16:31:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team has resolved the performance degradation issue on the NA6 instance. The problem began at 15:44 UTC and was resolved as of 15:56 UTC. During this time customers may have experienced slow performance while accessing the Salesforce application. Thank you for your continued patience. Mon, 13 May 2013 16:20:00 GMT http://trust.salesforce.com/trust/status 2013-05-13T16:20:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team is working to isolate a performance degradation issue on the NA6 instance. Please check the status of trust.salesforce.com frequently for updates regarding this issue. Mon, 13 May 2013 15:57:00 GMT http://trust.salesforce.com/trust/status 2013-05-13T15:57:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team has resolved the performance degradation issue on the NA6 instance. The problem began at 11:47 UTC and was resolved as of 15:25 UTC. During this time customers may have experienced slow performance while accessing the Salesforce application. Thank you for your continued patience. Mon, 13 May 2013 15:35:00 GMT http://trust.salesforce.com/trust/status 2013-05-13T15:35:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team continues to work on isolating the performance degradation issue on the NA6 instance. Customers may continue to see slowness or intermittent errors while accessing the Salesforce application at this time. Please check the status of trust.salesforce.com frequently for updates regarding this issue. Mon, 13 May 2013 14:30:00 GMT http://trust.salesforce.com/trust/status 2013-05-13T14:30:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team continues to work on isolating the performance degradation issue on the NA6 instance. Please check the status of trust.salesforce.com frequently for updates regarding this issue. Mon, 13 May 2013 13:55:00 GMT http://trust.salesforce.com/trust/status 2013-05-13T13:55:00Z http://trust.salesforce.com/trust/status The salesforce.com Technology Team is working to isolate a performance degradation issue on the NA6 instance. Please check the status of trust.salesforce.com frequently for updates regarding this issue. Mon, 13 May 2013 13:22:00 GMT http://trust.salesforce.com/trust/status 2013-05-13T13:22:00Z http://trust.salesforce.com/trust/status After a routine change was applied to the NA6 database tier, anomalous behavior was observed with the database tier's balancing of requests across nodes in the cluster. This behavior caused intermittent periods of resource contention and in turn degraded performance of customer requests. Several attempts were made to restore database stability without requiring customer downtime, but these efforts were not successful. On Tuesday May 14 at 7:00pm PDT, an emergency maintenance was executed to fully restart the NA6 database tier. After this maintenance activity, NA6 performance has been within normal parameters. Actions to Prevent Future Incidents: The salesforce.com Technology Team - - Is improving database maintenance procedures to reduce the chance of introducing this anomalous behavior - Is continuing to investigate the anomalous behavior with the database vendor Tue, 14 May 2013 23:59:00 GMT http://trust.salesforce.com/trust/status 2013-05-14T23:59:00Z http://trust.salesforce.com/trust/status After a routine change was applied to the NA6 database tier, anomalous behavior was observed with the database tier's balancing of requests across nodes in the cluster. This behavior caused intermittent periods of resource contention and in turn degraded performance of customer requests. Several attempts were made to restore database stability without requiring customer downtime, but these efforts were not successful. On Tuesday May 14 at 7:00pm PDT, an emergency maintenance was executed to fully restart the NA6 database tier. After this maintenance activity, NA6 performance has been within normal parameters. Actions to Prevent Future Incidents: The salesforce.com Technology Team - - Is improving database maintenance procedures to reduce the chance of introducing this anomalous behavior - Is continuing to investigate the anomalous behavior with the database vendor Mon, 13 May 2013 23:45:00 GMT http://trust.salesforce.com/trust/status 2013-05-13T23:45:00Z