Changes for page Topic Alerting

Last modified by Erik Bakker on 2025/11/19 20:30

From 40.5 to 40.4 From 66.1 to 65.1

From version 65.1

edited by Erik Bakker
on 2025/09/15 11:38

Change comment: There is no comment for this version

To version 40.5

edited by dfirdausy
on 2024/07/18 13:39

Change comment: There is no comment for this version

Raw
Rendered

Summary

Page properties (3 modified, 0 added, 0 removed)

Details

Page properties

Title

@@ -1,1 +1,1 @@
--Topic Alerting
++Customize Queue Alerting

Author

@@ -1,1 +1,1 @@
--XWiki.ebakker
++XWiki.dfirdausy

Content

@@ -1,93 +1,164 @@
  {{container}}{{container layoutStyle="columns"}}(((
--In this microlearning, we’ll explore how to view and customize topic alerts within your eMagiz environment. Customizing these alerts allows you to tailor the trigger conditions and thresholds to better fit your specific needs, enhancing how you monitor and respond to topic activity. We’ll cover the two types of topic triggers available and guide you through configuring them to suit your requirements. This will help you ensure your alerts are as effective and relevant as possible. Should you want to learn more about the standard alerting that eMagiz offers, please check out this [[microlearning>>doc:Main.eMagiz Academy.Microlearnings.Crash Course.Crash Course Platform.crashcourse-platform-manage-alerting-gen3||target="blank"]].
++In this microlearning, we will focus on how you can customize your queue alerts by configuring the trigger conditions and thresholds, which raise an alert and notify people when those points are passed. Should you want to learn more about the standard alerting that eMagiz offers, please check out this [[microlearning>>doc:Main.eMagiz Academy.Microlearnings.Crash Course.Crash Course Platform.crashcourse-platform-manage-alerting-gen3||target="blank"]].
  Should you have any questions, please contact [[academy@emagiz.com>>mailto:academy@emagiz.com]].
--{{info}}Note that this documentation explains the latest design of eMagiz Topic Alerting that is introduced in [[Release 253>>Main.Release Information.Release Blogs.253 - Changing Chameleon||target="blank"]] and the Topic Alerting explained in this page is only applicable for eMagiz models that are running on DSH.{{/info}}
--
  == 1. Prerequisites ==
  * Basic knowledge of the eMagiz platform
--* Basic knowledge of alerting in the eMagiz platform
  == 2. Key concepts ==
--This microlearning focuses on customizing the topic alerts that are existing in each of the environment of your model (i.e., Testing-Acceptance-Production). By customizing we mean further configuring the trigger conditions and thresholds of your topics, so they better suit your needs and requirements.
++This microlearning focuses on customizing the queue alerts for each environment in your model. By customizing: we mean further configuring the trigger conditions and thresholds of your queues beyond the general default settings provided by eMagiz.
--* eMagiz applies the same set of triggers to all topic.
--* There are two topic trigger options that eMagiz currently provides:
--** Minimum throughput check
--** Messages behind check
--* Recipients are defined on the topic level.
++* There are five trigger options
++** Error message
++** Log message
++** Queue consumers
++** Messages in queue
++** Queue throughput
++* On the trigger level, the recipients are defined
++* On the trigger level, the congestion control is defined
--== 3. Topic Alerting ==
++== 3. Dynamic Alerting ==
--As all our alerting now works is that we have divided them up in various segments that are accessible via Manage.
++To enhance the observability of your integration landscape while running in the 3rd generation runtime architecture, we have added a new feature to our Manage phase called "Dynamic Alerts." You can access this functionality via the "Alerting" menu in Manage. Then, with the help of the "Trigger" overview, you can view all triggers in your environment. On the top of the list, you will see all [["static" alerts>>doc:Main.eMagiz Academy.Microlearnings.Crash Course.Crash Course Platform.crashcourse-platform-manage-alerting-gen3||target="blank"]] as defined by eMagiz below that you will see all "dynamic" alerts that you and your fellow team members (with sufficient rights) can view, edit and delete.
--[[image:Main.Images.Microlearning.WebHome@crashcourse-platform-manage-alerting-in-emagiz-topic-alerts-overview.png]]
++We offer alerting on seven types.
--Here, when you navigate to Manage->Alerting->Triggers, you will find all the topics that are generated by eMagiz currently existing within your model, along with the trigger conditions and thresholds that are applied to each of these topics. In the following sections, we will discuss how you can tune these topics triggers to your needs.
++* Error message
++* Log message
++* Queue consumers
++* Messages in queue
++* Queue throughput
++* Messages added to topic
++* Topic consumer messages behind
--{{info}}Contrary to the queue alerting we do not offer any default configuration as eMagiz given the widespread implementation of event streaming in the community which prevents us from defining relevant default triggers that would apply in almost all cases. As a result, the default configuration on topic level is that the alert is **disabled** and **no** thresholds are defined{{/info}}
++[[image:Main.Images.Microlearning.WebHome@novice-alerting-dynamic-alerting--dynamic-alert-trigger-options.png]]
--=== 3.1 Customize Default Values ===
++First, we will discuss the configuration of each trigger type. Subsequently, we will discuss defining which recipients will be notified and how to alter the default congestion control settings (as this is the same for each trigger type).
--Changes to the default values will only apply to topic triggers that are set to use the default values, and not to those you have already customized. To modify the default values configured by eMagiz, you can click the "Configure defaults" button at the top of the topic trigger list.
++=== 3.1 Error message ===
--{{info}}Note that if your account does not have edit rights in Manage for a specific environment, you will see a "View defaults" button instead.{{/info}}
++On top of the "static" alert that is autogenerated by eMagiz on Error messages, you can add additional "dynamic" alerts that only trigger when an error message is raised for which a specific header contains a particular value. This is most likely relevant when a specific error message with certain characteristics (i.e., a particular header value) needs to be sent to a different set of recipients.
--[[image:Main.Images.Microlearning.WebHome@novice-alerting-customize-topic-alerting--customize-default-triggers.png]]
++To define the trigger, fill in the header's name and the expected value. On top of that, you can fill in a description that will be added to the alert received via email. Once this is filled in, the configuration could look as follows.
--Here, you will get a pop-up page where you can configure the default values of your topic triggers. The details of each of the triggers will be discussed in the next section. But, what is important to note here is that, here, you have the "Reset to eMagiz defaults" button to reset the default values to the values provided by eMagiz as defaults (i.e. disabling them). Next to that, at the bottom of the pop-up, you will find the "Update defaults" button, which will apply the default values that you have configured to all topic triggers that are set to use the default values. By clicking this button, the new default values will be put into effect immediately.
++[[image:Main.Images.Microlearning.WebHome@novice-alerting-dynamic-alerting--dynamic-alert-error-message-configuration.png]]
--Additionally, on this page, you can also find the "Notifications" tab that allows you to configure which recipients that you want to receive the notifications from the alerting. If you want to learn more about this notifications and how to configure it, please refer to this [[microlearning>>Main.eMagiz Academy.Microlearnings.Intermediate Level.Alerting.intermediate-alerting-setting-up-notifications-gen3||target="blank"]].
++=== 3.2 Log message ===
--=== 3.2 Customize Topic Alerting ===
++On top of the "static" alert that is autogenerated by eMagiz on Log messages, you can add additional "dynamic" alerts that only trigger when a log message contains a specific phrase. This is likely when you log something on which you want to alert when it happens.
--Next to customizing the default values for all topic triggers, you can also modify the trigger configurations for individual topics. To customize these topic triggers to better suit your needs, you can click the edit buttons (green paper and pencil buttons) located at the end of each topic trigger. By doing so, you will get a pop-up page that allows you to further customize the conditions and thresholds of your topic triggers, which will be discussed in the following.
++To define the trigger, fill in the phrase you want to check. On top of that, you can fill in a description that will be added to the alert received via email. Once this is filled in, the configuration could look as follows.
--==== 3.2.1 Minimum throughput check ====
--[[image:Main.Images.Microlearning.WebHome@novice-alerting-customize-queue-alerting-consumers-check-queue.png]]
++[[image:Main.Images.Microlearning.WebHome@novice-alerting-dynamic-alerting--dynamic-alert-log-message-configuration.png]]
--{{info}}This trigger checks how many processes (consumers) are subscribing to and retrieving messages from a particular queue over the last ten minutes. It evaluates whether the number of consumers meets the specified threshold. If the number of consumers falls consistently **below** the "Expected consumers" value for a period of ten minutes, an alert is generated. You can customize this threshold as needed. Additionally, you can specify the instructions to be included in email notifications sent to recipients and in the alert message displayed in the Manage->Alerting->Alerts.{{/info}}
++=== 3.3 Queue consumers ===
--* **Queue load check**
--[[image:Main.Images.Microlearning.WebHome@novice-alerting-customize-queue-alerting-queue-load-check.png]]
++On top of the "static" alert that is autogenerated by eMagiz on Queue consumers, you can add additional "dynamic" alerts that only trigger when the number of consumers on a (specific) set of queues is below a certain threshold.
--{{info}}This trigger checks how many messages are produced on a particular queue and not yet retrieved by any consumers over the last ten minutes. It measures the number of messages on the queue. If the number of messages is consistently **greater** than the threshold value (i.e., "Max messages queued") for a period of ten minutes, an alert is generated. Otherwise, no alert will be generated. You can customize this threshold as needed. Additionally, you can specify the instructions to be included in email notifications sent to recipients and in the alert message displayed in the Manage->Alerting->Alerts. It is important to note that if high bursts of messages occur regularly (e.g., during nightly batch processing, 1000 messages are expected to be produced to a certain queue), then setting a higher threshold (e.g., 1000 Max messages queued) can prevent false-positive alerts caused by the default configuration.{{/info}}
++To define the trigger, determine the threshold you want to check (i.e., 2). Subsequently, specify whether on which queues (or all queues) you want eMagiz to check for this condition. No further action is needed if you want to check on all queues.
--* **Inactivity check**
--[[image:Main.Images.Microlearning.WebHome@novice-alerting-customize-queue-alerting-inactivity-check.png]]
++[[image:Main.Images.Microlearning.WebHome@novice-alerting-dynamic-alerting--dynamic-alert-queue-consumers-all-queues.png]]
--{{info}}This trigger checks whether a particular queue receives at least one message regularly or within the specified time frame. With this trigger, you specify whether you want to 'always' have the check running everyday or only during a specified time range. If the queue receives a message within a specified time frame (i.e., time frame specified in the 'Expect a message every'), then the timer will reset and the next evaluation interval will start. Therefore, this trigger can produce two or more alerts a day if that setting is set to less than half a day. Similar to the other triggers, you can specify the instructions to be included in email notifications sent to recipients and in the alert message displayed in the Manage->Alerting->Alerts.
--* If you select the "Always" option, the next step is to configure the "Expect a message every" period. This is useful if you want to run the check every day and be notified when your queue has not received at least one message within a specified time frame, such as within an hour, a day or more.
--* If you select the "During time range" option, you can specify the days of the week and the starting time for the trigger to evaluate the queue. You will also need to specify the duration for which the trigger will evaluate and how frequently within that duration the queue should expect at least one message. Note that for this option, the "Expect a message every" setting must allow for at least two occurrences within the specified duration.
--* In the example above, the trigger is set to evaluate from Monday to Friday, 9:00 to 17:00, and expects a message every four hours, an alert will be sent if no message is received by 13:00. However, if a message is received at 11:00, then the timer resets, and the next check will be at 15:00. If no message is received by then, an alert will be sent. Consequently, due to the configuration of expects a message every four hours, and considering that there are only two hours left until the configured 17:00, the trigger for the remaining two hours will not be executed.{{/info}}
++You can uncheck the checkbox if you want to check on a specific (set of) queue(s). As a result, the "Add" and "Remove" buttons are displayed by eMagiz, through which you can add and remove queues to the trigger.
--* **Periodic activity check**
--  [[image:Main.Images.Microlearning.WebHome@novice-alerting-customize-queue-alerting-periodic-activity-check.png]]
++[[image:Main.Images.Microlearning.WebHome@novice-alerting-dynamic-alerting--dynamic-alert-queue-consumers-specific-queues.png]]
--{{info}}This trigger checks whether at least the specified number of messages is received during the specified interval. Similar to the others, you can also specify the instructions to be included in email notifications sent to recipients and in the alert message displayed in Manage->Alerting->Alerts.
--* If you enable this trigger, you must also specify the days of the week, the starting time, and the duration for the trigger to evaluate the queue. Additionally, you need to specify the minimum number of messages the queue is expected to receive. If this number is **not** reached, an alert will be generated.
--* Unlike the Inactivity check, this Periodic Activity check evaluates only at **the end of each interval** and does **not** reset the timer when the minimum number of messages is reached during the interval. This means that this trigger will not produce more than one alert per day.
--* In the example above, the trigger is set to evaluate from Monday to Friday, from 9:00 to 8:45 in the following morning, and expects a minimum of 100 messages within each interval. If the 100 messages are not received by that 8:45 in the next morning, an alert will be sent. Alternatively, if the 100 messages are reached, no alert will be generated for that interval, and the next check will start again at 9:00. For intervals starting on Friday, the next check will begin at 9:00 AM on Monday, as Saturday and Sunday are not included.{{/info}}
++When you press the "Add" button, a new pop-up will appear through which you can select the queues for which you want the trigger to be activated. In this pop-up, you can choose one (or more) queues and press the "Select" button to add them to the trigger.
--Upon saving, these custom values will apply immediately, putting the new check into effect. Additionally, you will notice that the "Settings" status of the queue trigger you just customized has changed from "Default" to "Custom". Similarly, on this page, you can also find the "Notifications" tab that allows you to configure which recipients that you want to receive the notifications from the alerting of a specific queue. If you want to learn more about this notifications and how to configure it, please refer to this [[microlearning>>Main.eMagiz Academy.Microlearnings.Intermediate Level.Alerting.intermediate-alerting-setting-up-notifications-gen3||target="blank"]].
++[[image:Main.Images.Microlearning.WebHome@novice-alerting-dynamic-alerting--dynamic-alert-queue-consumers-queue-selection.png]]
++{{info}}Note that when you want to select multiple queues at once, you cannot press "Enter" to Save your selection. In this case, you need to press the "Select" button to save your choice.{{/info}}
++
++Once you have selected the queue(s) and pressed "Select," eMagiz will close the pop-up and add the queues to the trigger.
++
++[[image:Main.Images.Microlearning.WebHome@novice-alerting-dynamic-alerting--dynamic-alert-queue-consumers-queue-selection-result.png]]
++
++=== 3.4 Messages in queue ===
++
++On top of the "static" alert that is autogenerated by eMagiz on Messages in a queue, you can add additional "dynamic" alerts that only trigger when a queue holds more than X messages. Logically, the X can be determined by the user. Subsequently, define whether on which queues (or all queues) you want eMagiz to check for this condition. No further action is needed if you want to check on all queues.
++
++[[image:Main.Images.Microlearning.WebHome@novice-alerting-dynamic-alerting--dynamic-alert-messages-in-queue-all-queues.png]]
++
++You can uncheck the checkbox if you want to check on a specific (set of) queue(s). As a result, the "Add" and "Remove" buttons are displayed by eMagiz, through which you can add and remove queues to the trigger.
++
++{{info}}This process of adding and removing specific queues works identically, as explained in the section on Queue consumers.[[Queue consumers.>>||anchor="H3.3Queueconsumers"]]{{/info}}
++
++Once you have selected the queue(s) and pressed "Select," eMagiz will close the pop-up and add the queues to the trigger.
++
++[[image:Main.Images.Microlearning.WebHome@novice-alerting-dynamic-alerting--dynamic-alert-messages-in-queue-queue-selection-result.png]]
++
++=== 3.5 Queue througput ===
++
++For this category you can add additional "dynamic" alerts that only trigger when the throughput on one or more queues is more or less than X messages per Y minutes. Subsequently, define whether on which queues (or all queues) you want eMagiz to check for this condition. No further action is needed if you want to check on all queues.
++
++[[image:Main.Images.Microlearning.WebHome@novice-alerting-dynamic-alerting--dynamic-alert-queue-throughput-all-queues.png]]
++
++You can uncheck the checkbox if you want to check on a specific (set of) queue(s). As a result, the "Add" and "Remove" buttons are displayed by eMagiz, through which you can add and remove queues to the trigger.
++
++{{info}}This process of adding and removing specific queues works identically, as explained in the section on Queue consumers.[[Queue consumers.>>||anchor="H3.3Queueconsumers"]]{{/info}}
++
++For this trigger, additional criteria can be configured that allow you to define when eMagiz needs to validate this configuration (i.e., Occurence). With the help of this configuration option, you can specify when the trigger needs to be evaluated. An example could be only on Monday between 04:00 AM and 05:00 AM.
++
++[[image:Main.Images.Microlearning.WebHome@novice-alerting-dynamic-alerting--dynamic-alert-queue-throughput-occurence-config.png]]
++
++=== 3.6 Messages added to topic ===
++
++For this category, you can add additional "dynamic" alerts. These alerts only trigger when the messages added to one or more topic is less than X messages per Y minutes. On top of the number of messages (X) and the timeframe (Y), you can define whether to validate this on a specific topic or all topics. Note that the default configuration is that the alert applies to all topics.
++
++[[image:Main.Images.Microlearning.WebHome@novice-alerting-dynamic-alerting--dynamic-alert-messages-added-to-topic-all-topics.png]]
++
++You can uncheck the checkbox to check on a specific (set of) topic(s). As a result, the "Add" and "Remove" buttons are displayed by eMagiz, through which you can add and remove topics to the trigger.
++
++{{info}}This process of adding and removing specific topics works identically, as explained in the section on Queue consumers.[[Queue consumers.>>||anchor="H3.3Queueconsumers"]]{{/info}}
++
++For this trigger, additional criteria can be configured that allow you to define when eMagiz needs to validate this configuration (i.e., Occurence). With the help of this configuration option, you can specify when the trigger needs to be evaluated. An example could be only on Monday between 04:00 AM and 05:00 AM.
++
++=== 3.7 Topic consumer lag ===
++
++For this category, you can add additional "dynamic" alerts. These alerts only trigger when one (or more) consumer groups lag by X messages on one (or more) topic(s). Note that the default configuration is that you, as a user, need to specify for which topic(s) the alert applies, as consumers' behavior can vary widely.
++
++{{info}}This process of adding and removing specific topics works identically, as explained in the section on Queue consumers.[[Queue consumers.>>||anchor="H3.3Queueconsumers"]]{{/info}}
++
++Once filled in, the solution should look similar as follows.
++
++[[image:Main.Images.Microlearning.WebHome@novice-alerting-dynamic-alerting--dynamic-alert-topic-consumer-lag-selected-topics.png]]
++
++=== 3.8 Recipient(s) configuration ===
++
++You can select your recipients when you switch from the "Trigger" to the "Output" tab. In this overview, you can choose both eMagiz users and external recipients.
++
++[[image:Main.Images.Microlearning.WebHome@novice-alerting-dynamic-alerting--dynamic-alert-recipient-config.png]]
++
++{{info}}Note that when you define multiple external recipients, you need to separate each email address with a comma to make it work.{{/info}}
++
++=== 3.9 Congestion control configuration ===
++
++In the same "Output" tab, you can also define the congestion control configuration. The default configuration is that congestion control will be activated once more than ten alerts are triggered in the last ten minutes. So should you want to reduce this number, you can do so here.
++
++[[image:Main.Images.Microlearning.WebHome@novice-alerting-dynamic-alerting--dynamic-alert-congestion-control-config.png]]
++
++{{warning}}Note that we advise only reducing the number under the congestion control if the ten default is too high. We do **not** advise deactivating the congestion control as this can cause an explosion in the number of received notifications.{{/warning}}
++
  == 4. Key takeaways ==
--* As a user with the edit rights in Manage of a particular environment, you can modify the default values for all of your queue triggers or customize the trigger configurations of individual queues.
--* eMagiz provides four queue trigger options: Consumers Check (monitoring the number of processes consuming messages), Queue Load Check (monitoring the number of queued messages), Inactivity Check (ensuring regular received message), and Periodic Activity Check (ensuring a minimum number of messages within a specified interval).
--* Once you save your trigger configurations, they will take effect immediately.
++* There are seven trigger options
++** Error message
++** Log message
++** Queue consumers
++** Messages in queue
++** Queue throughput
++** Messages added to topic
++** Topic consumer messages behind
++* On the trigger level, the recipients are defined
++* On the trigger level, the congestion control is defined
++* Default configuration of the congestion control is sufficient in most cases
++* When entering multiple external recipients, ensure each email address is separated with a comma
  == 5. Suggested Additional Readings ==
--If you are interested in this topic and want more information, please read the help text provided by eMagiz and read the following microlearning.
--* [[Crash Courses (Menu)>>doc:Main.eMagiz Academy.Microlearnings.Crash Course.WebHome||target="blank"]]
--** [[Crash Course Platform (Navigation)>>doc:Main.eMagiz Academy.Microlearnings.Crash Course.Crash Course Platform.WebHome||target="blank"]]
--*** [[Standard Alerting eMagiz (Explanation)>>doc:Main.eMagiz Academy.Microlearnings.Crash Course.Crash Course Platform.crashcourse-platform-manage-alerting-gen3||target="blank"]]
--* [[Intermediate (Menu)>>doc:Main.eMagiz Academy.Microlearnings.Intermediate Level.WebHome||target="blank"]]
--** [[Alerting (Navigation)>>doc:Main.eMagiz Academy.Microlearnings.Intermediate Level.Alerting.WebHome||target="blank"]]
--*** [[Setting up notifications (Explanation)>>doc:Main.eMagiz Academy.Microlearnings.Intermediate Level.Alerting.intermediate-alerting-setting-up-notifications-gen3||target="blank"]]
--* [[Queue Alerting (Search Result)>>url:https://docs.emagiz.com/bin/view/Main/Search?sort=score&sortOrder=desc&highlight=true&facet=true&r=1&f_space_facet=0%2FMain.&f_type=DOCUMENT&f_locale=en&f_locale=&f_locale=en&text=%22queue+alerting%22||target="blank"]]
++If you are interested in this topic and want more information, please read the help text provided by eMagiz.
++
  )))((({{toc/}}))){{/container}}{{/container}}