Disaster Recovery

Backup and Recovery

  1. How is the data backup strategy organized?

    VoiceConsole SaaS deployment uses Azure SQL Platform as a service and is configured as a Business-Critical tier. Additionally, VoiceConsole SaaS deployment uses Zone Redundant Storage (ZRS) to maintain high availability. Within the ZRS structure, data is replicated across the available zones to maintain reliability. If a zone becomes unavailable for any reason, there is no delay to remount Azure file shares for any connected client. Data backups of file shares occur regularly. A full backup occurs once per day and that backup is maintained for ten days. This tier model uses a cluster of database engine processes so that there are always available database engine nodes even during maintenance activities.

  2. How is data storage structured to facilitate disaster recovery?

    High availability is achieved by the replication of data between nodes, with each node using an attached Solid-State Drive for data storage. The deployment also features an added layer of data protection through the failover strategy. Should a failure occur in one region, failover is set up in another region to maintain data availability. Outside of the EU, the system architecture features the primary server located in the US East and a Secondary server located in US West (Passive/Failover). Within the EU, the system architecture features the primary server located in The Netherlands and a Secondary server located in Ireland (Passive/Failover).

  3. What is the frequency of data backups?

    VoiceConsole SaaS deployment offers a variety of backup instances and retention policies based on the data type and storage capabilities.

    Database backups include short-term and long-term policies. The short-term policy retains a backup every 12 hours and retains the data for seven days. Long-term backups occur both weekly (retained for 12 weeks) and monthly (retained for three months). Regular backups are in place for both cluster Azure Kubernetes Service (AKS) and Database-Backend Services. AKS Backups via Azure occur every four hours and are retained for seven days. Azure storage account backups occur once per day and are retained for 10 days. Fileshare SoftDelete is retained for 14 days. KeyVault storage is retained for 90 days.

    To maintain data availability, regular backups occur for the applicable failover mechanism.

  4. What are the communication protocols for Planned Maintenance?

    When Planned Maintenance requires downtime, customers are notified via email at least 30 days in advance prior to the maintenance period. If Planned Maintenance does not require downtime, notice is deployed 15 days in advance.

  5. What are the communication protocols for patches, infrastructure upgrades, and continuous integrations?

    Separately, patch releases are mandatory upon release with no UAT environment meaning all customers adhere to the same patch release schedule.

    Regarding infrastructure upgrades, Honeywell makes every effort to schedule upgrades during a low activity time period across all timezones. Infrastructure updates are mandatory and cannot be delayed, however, downtime duration is typically low or not present. Infrastructure upgrades do not have a UAT environment and do not count against the uptime Service-Level Agreement (SLA).

    Server production breaking changes will be communicated to customers via email at least six (6) months in advance prior to taking place. While Honeywell makes every effort to provide customers advanced of breaking changes, there are certain emergency cases which may employ a shorter advanced notice procedure given the critical nature of the change.

  6. What are the communication protocols when there is service degradation or outage?

    Should a service interruption occur, all Critical Service Alerts are routed to ServiceNow and mapped to respective Continuous Integration and Support groups for resolution. These groups immediately notify customers through Honeywell's ticketing system. Customers are also notified of Unplanned Service interruptions via email as soon as possible.

 

Monitoring

  1. How are systems monitored?

    Honeywell monitors Voice Console SaaS deployment at both the application level and infrastructure level to track availability, endpoint response time, and service alerts.

    At the application level:

    • Each customer instance is monitored via custom health APIs exposed by VoiceConsole SaaS deployment

    • Elastic APM (Application Performance Monitoring) is integrated for in-depth application level monitoring and alerting

    At the infrastructure level:

    • Service Level Objectives (SLO) are defined to monitor critical infrastructure and provide appropriate alerts for:

      • AKS (Azure Kubernetes Service) clusters

      • Storage account

      • Database servers

 

Reporting Problems

  1. How are problems detected by Honeywell reported?

    If Honeywell detects an issue, an investigation is performed and customers are kept informed during the process through resolution update and confirmation messages.

  2. How are customer issues reported?

    If a customer reports an issue, Honeywell investigates and escalates as needed. Any necessary improvements are implemented.

  3. What response methods and times can Honeywell deliver?

    While Honeywell communicates important information via the VoiceConsole SaaS deployment interface, it is important that Honeywell have updated contact information in order to provide email notices of system maintenance, planned upgrades, and other important events. Please ensure that contact information is current and that any VoiceConsole SaaS deployment email messages are not getting caught in your spam filter.

  4. Who is responsible in the event Azure has a problem?

    If Azure has a problem or Service Outage, PSSOperations will be responsible.