* Redfish EventService Implementation @ 2020-01-31 20:53 RAJESWARAN THILLAIGOVINDAN 2020-02-09 20:22 ` RAJESWARAN THILLAIGOVINDAN 2020-06-08 21:08 ` Redfish EventService Implementation Brad Bishop 0 siblings, 2 replies; 44+ messages in thread From: RAJESWARAN THILLAIGOVINDAN @ 2020-01-31 20:53 UTC (permalink / raw) To: openbmc, apparao.puli Hi, I am going through the bmcweb code for implementing Redfish EventService based on the design document https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design is hooked to the journal based Redfish Event Logging. For life cycle events(ResourceAdded, ResourceRemoved, ResourceUpdated), using D-Bus match, bmcweb can create an event log. This requires a JSON dictionary, comprising an array of Redfish Resource Name and the D-Bus path. This approach works only in case of one to one mapping of Redfish Resource Name and the D-Bus path. For propertiesChanged events, if the Redfish Resource property is not on the same D-Bus path or the Redfish Resource property name is different from the D-Bus property name, then an additional JSON dictionary to maintain this information is required. With D-Bus match alone in the bmcweb, Redfish EventService can't be fully supported. For the Message Registers and the Resource Types that are supported, the relevant OpenBMC application must create an event log in the journal using either the phosphor::logging::entry or sd_journal_send() command. After realizing that with D-Bus match in the bmcweb alone can't help to fully implement EventService, I prefer to avoid using D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC application that generated the event to create an event log in the journal. Do you see any advantage of using combination of D-Bus match in the bmcweb wherever it is possible and changes to OpenBMC application in other cases to create an event log ? Your views are highly appreciated. Thanks, Rajes ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-01-31 20:53 Redfish EventService Implementation RAJESWARAN THILLAIGOVINDAN @ 2020-02-09 20:22 ` RAJESWARAN THILLAIGOVINDAN 2020-02-17 20:11 ` RAJESWARAN THILLAIGOVINDAN 2020-02-19 19:19 ` Puli, Apparao 2020-06-08 21:08 ` Redfish EventService Implementation Brad Bishop 1 sibling, 2 replies; 44+ messages in thread From: RAJESWARAN THILLAIGOVINDAN @ 2020-02-09 20:22 UTC (permalink / raw) To: openbmc, apparao.puli, James Feist ApparaRao. As you have shown interest in this feature and submitted the design document, do you have any opinion on this? Do you see any merit in using D-Bus match in bmcweb to create event logs for life cycle events? Please feel free to weigh in. Thanks, Rajes On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote: > Hi, > > I am going through the bmcweb code for implementing Redfish > EventService based on the design document > https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design > is hooked to the journal based Redfish Event Logging. For life cycle > events(ResourceAdded, ResourceRemoved, ResourceUpdated), using D-Bus > match, bmcweb can create an event log. This requires a JSON > dictionary, comprising an array of Redfish Resource Name and the D-Bus > path. This approach works only in case of one to one mapping of > Redfish Resource Name and the D-Bus path. For propertiesChanged > events, if the Redfish Resource property is not on the same D-Bus path > or the Redfish Resource property name is different from the D-Bus > property name, then an additional JSON dictionary to maintain this > information is required. With D-Bus match alone in the bmcweb, Redfish > EventService can't be fully supported. For the Message Registers and > the Resource Types that are supported, the relevant OpenBMC > application must create an event log in the journal using either the > phosphor::logging::entry or sd_journal_send() command. > > After realizing that with D-Bus match in the bmcweb alone can't help > to fully implement EventService, I prefer to avoid using D-Bus match > in bmcweb. Instead, I prefer to modify the OpenBMC application that > generated the event to create an event log in the journal. Do you see > any advantage of using combination of D-Bus match in the bmcweb > wherever it is possible and changes to OpenBMC application in other > cases to create an event log ? > > Your views are highly appreciated. > > Thanks, > Rajes ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-02-09 20:22 ` RAJESWARAN THILLAIGOVINDAN @ 2020-02-17 20:11 ` RAJESWARAN THILLAIGOVINDAN 2020-02-19 19:19 ` Puli, Apparao 1 sibling, 0 replies; 44+ messages in thread From: RAJESWARAN THILLAIGOVINDAN @ 2020-02-17 20:11 UTC (permalink / raw) To: openbmc I have implemented the skeleton code and uploaded it for review. Kindly review and let me know your comments. For prototyping, based on the Redfish Event Logging design, I have modified phosphosr-user-manager application to log resource creation event when an account is created. For reading the redfish event logs from the journal and writing to BMC filesystem(/var/log/redfish), I have pulled the rsyslog configuration from the OEM (Intel) and made minor changes. These patches are work in progress and needs lot more changes. Any suggestions regarding the approach is appreciated. https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/29464 https://gerrit.openbmc-project.xyz/c/openbmc/phosphor-user-manager/+/29463 https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/29462 Thanks, Rajes On 10-02-2020 01:52, RAJESWARAN THILLAIGOVINDAN wrote: > ApparaRao. > > As you have shown interest in this feature and submitted the design > document, do you have any opinion on this? Do you see any merit in > using D-Bus match in bmcweb to create event logs for life cycle > events? Please feel free to weigh in. > > Thanks, > Rajes > > On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote: >> Hi, >> >> I am going through the bmcweb code for implementing Redfish >> EventService based on the design document >> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This >> design is hooked to the journal based Redfish Event Logging. For life >> cycle events(ResourceAdded, ResourceRemoved, ResourceUpdated), using >> D-Bus match, bmcweb can create an event log. This requires a JSON >> dictionary, comprising an array of Redfish Resource Name and the >> D-Bus path. This approach works only in case of one to one mapping of >> Redfish Resource Name and the D-Bus path. For propertiesChanged >> events, if the Redfish Resource property is not on the same D-Bus >> path or the Redfish Resource property name is different from the >> D-Bus property name, then an additional JSON dictionary to maintain >> this information is required. With D-Bus match alone in the bmcweb, >> Redfish EventService can't be fully supported. For the Message >> Registers and the Resource Types that are supported, the relevant >> OpenBMC application must create an event log in the journal using >> either the phosphor::logging::entry or sd_journal_send() command. >> >> After realizing that with D-Bus match in the bmcweb alone can't help >> to fully implement EventService, I prefer to avoid using D-Bus match >> in bmcweb. Instead, I prefer to modify the OpenBMC application that >> generated the event to create an event log in the journal. Do you see >> any advantage of using combination of D-Bus match in the bmcweb >> wherever it is possible and changes to OpenBMC application in other >> cases to create an event log ? >> >> Your views are highly appreciated. >> >> Thanks, >> Rajes > ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-02-09 20:22 ` RAJESWARAN THILLAIGOVINDAN 2020-02-17 20:11 ` RAJESWARAN THILLAIGOVINDAN @ 2020-02-19 19:19 ` Puli, Apparao 2020-02-24 6:37 ` Ratan Gupta 1 sibling, 1 reply; 44+ messages in thread From: Puli, Apparao @ 2020-02-19 19:19 UTC (permalink / raw) To: RAJESWARAN THILLAIGOVINDAN, openbmc, James Feist Hi, I am sorry for late response as this mail is buried under and got struck back of my mind. As i did mentioned in EventService design document, EventLog Monitor service is not common across the organizations( Example: Intel uses the redfish event logs file and inotify mechanism for monitoring the event logs. Where as IBM uses d-bus event log mechanism and they can relay on match rules). That said challenges with ResourceType mapping will be different in both monitoring mechanisms. This is good point. Initially when i started EventService design, i thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As per Intel design) but not sure that may become difficult when we expand supported ResourceTypes. As per my reading from below query, You are looking at d-bus match rules and ResourceTypes mapping which is more specific to d-bus event logging(IBM way of implementing event logging). reading it from journal logs will give more information but that will impact the performance to large extent. This might be one of the reason why we (Intel) uses Redfish message ID while logging redfish events logs to journal(You can refer design document for same at https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). In opinion, in your d-bus if you are using some kind of filter(Example REDFISH_MESSAGE_ID) while logging in journal logs for all events and figure out the way to monitor the journal logs without impacting the performance, that should be ok as long as match filters are satisfied for Redfish EventService subscriptions and supported Types(Again differs with implementation). Thanks, -Appu On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote: > ApparaRao. > > As you have shown interest in this feature and submitted the design > document, do you have any opinion on this? Do you see any merit in > using D-Bus match in bmcweb to create event logs for life cycle > events? Please feel free to weigh in. > > Thanks, > Rajes > > On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote: >> Hi, >> >> I am going through the bmcweb code for implementing Redfish >> EventService based on the design document >> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This >> design is hooked to the journal based Redfish Event Logging. For life >> cycle events(ResourceAdded, ResourceRemoved, ResourceUpdated), using >> D-Bus match, bmcweb can create an event log. This requires a JSON >> dictionary, comprising an array of Redfish Resource Name and the >> D-Bus path. This approach works only in case of one to one mapping of >> Redfish Resource Name and the D-Bus path. For propertiesChanged >> events, if the Redfish Resource property is not on the same D-Bus >> path or the Redfish Resource property name is different from the >> D-Bus property name, then an additional JSON dictionary to maintain >> this information is required. With D-Bus match alone in the bmcweb, >> Redfish EventService can't be fully supported. For the Message >> Registers and the Resource Types that are supported, the relevant >> OpenBMC application must create an event log in the journal using >> either the phosphor::logging::entry or sd_journal_send() command. >> >> After realizing that with D-Bus match in the bmcweb alone can't help >> to fully implement EventService, I prefer to avoid using D-Bus match >> in bmcweb. Instead, I prefer to modify the OpenBMC application that >> generated the event to create an event log in the journal. Do you see >> any advantage of using combination of D-Bus match in the bmcweb >> wherever it is possible and changes to OpenBMC application in other >> cases to create an event log ? >> >> Your views are highly appreciated. >> >> Thanks, >> Rajes > ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-02-19 19:19 ` Puli, Apparao @ 2020-02-24 6:37 ` Ratan Gupta 2020-02-25 14:06 ` Puli, Apparao 0 siblings, 1 reply; 44+ messages in thread From: Ratan Gupta @ 2020-02-24 6:37 UTC (permalink / raw) To: openbmc, Puli, Apparao [-- Attachment #1: Type: text/plain, Size: 5454 bytes --] Hi Apparao, On 2/20/20 12:49 AM, Puli, Apparao wrote: > Hi, > > I am sorry for late response as this mail is buried under and got > struck back of my mind. > > As i did mentioned in EventService design document, EventLog Monitor > service is not common across the organizations( Example: Intel uses > the redfish event logs file and inotify mechanism for monitoring the > event logs. Where as IBM uses d-bus event log mechanism and they can > relay on match rules). That said challenges with ResourceType mapping > will be different in both monitoring mechanisms. This is good point. > Initially when i started EventService design, i thought we can have > mapping in bmcweb for ResourceTypes with set of MessageID's for Logged > events ( As per Intel design) but not sure that may become difficult > when we expand supported ResourceTypes. If I am getting correctly, Here is the flow which Intel uses. 1. Individual repo have to push the logs using sd_journal_send which will write to the file(/var/log/redfish) by using rsyslog daemon sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>, "REDFISH_MESSAGE_ID=%s", "ResourceEvent.1.0.ResourceCreated",NULL); * How you would populate the "OriginOfCondition" during sending of event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json) * Any plan to add resource path in this journal message which tells that this is the path for which resource created event generated. * Would the path be "Redfish Path/ D-bus Path"? * Where the mapping would be done between D-busPath/Redfish Resource path? Cons: Every application have to make the change(use sd_journal_send). My thought is backend application should not be aware of the redfish terminlogy. *2.* Some application(bmcweb) would do the Inotify on the path(/var/log/redfish) and send the event once there is any activity on this file. > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As per Intel design) Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type. > > As per my reading from below query, You are looking at d-bus match > rules and ResourceTypes mapping which is more specific to d-bus event > logging(IBM way of implementing event logging). reading it from > journal logs will give more information but that will impact the > performance to large extent. This might be one of the reason why we > (Intel) uses Redfish message ID while logging redfish events logs to > journal(You can refer design document for same at > https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). > In opinion, in your d-bus if you are using some kind of filter(Example > REDFISH_MESSAGE_ID) while logging in journal logs for all events and > figure out the way to monitor the journal logs without impacting the > performance, that should be ok as long as match filters are satisfied > for Redfish EventService subscriptions and supported Types(Again > differs with implementation). > > Thanks, > > -Appu > > On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote: >> ApparaRao. >> >> As you have shown interest in this feature and submitted the design >> document, do you have any opinion on this? Do you see any merit in >> using D-Bus match in bmcweb to create event logs for life cycle >> events? Please feel free to weigh in. >> >> Thanks, >> Rajes >> >> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote: >>> Hi, >>> >>> I am going through the bmcweb code for implementing Redfish >>> EventService based on the design document >>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This >>> design is hooked to the journal based Redfish Event Logging. For >>> life cycle events(ResourceAdded, ResourceRemoved, ResourceUpdated), >>> using D-Bus match, bmcweb can create an event log. This requires a >>> JSON dictionary, comprising an array of Redfish Resource Name and >>> the D-Bus path. This approach works only in case of one to one >>> mapping of Redfish Resource Name and the D-Bus path. For >>> propertiesChanged events, if the Redfish Resource property is not on >>> the same D-Bus path or the Redfish Resource property name is >>> different from the D-Bus property name, then an additional JSON >>> dictionary to maintain this information is required. With D-Bus >>> match alone in the bmcweb, Redfish EventService can't be fully >>> supported. For the Message Registers and the Resource Types that are >>> supported, the relevant OpenBMC application must create an event log >>> in the journal using either the phosphor::logging::entry or >>> sd_journal_send() command. >>> >>> After realizing that with D-Bus match in the bmcweb alone can't help >>> to fully implement EventService, I prefer to avoid using D-Bus match >>> in bmcweb. Instead, I prefer to modify the OpenBMC application that >>> generated the event to create an event log in the journal. Do you >>> see any advantage of using combination of D-Bus match in the bmcweb >>> wherever it is possible and changes to OpenBMC application in other >>> cases to create an event log ? >>> >>> Your views are highly appreciated. >>> >>> Thanks, >>> Rajes >> Thanks Ratan [-- Attachment #2: Type: text/html, Size: 8088 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-02-24 6:37 ` Ratan Gupta @ 2020-02-25 14:06 ` Puli, Apparao 2020-05-05 11:43 ` RAJESWARAN THILLAIGOVINDAN 2020-05-26 12:20 ` RAJESWARAN THILLAIGOVINDAN 0 siblings, 2 replies; 44+ messages in thread From: Puli, Apparao @ 2020-02-25 14:06 UTC (permalink / raw) To: Ratan Gupta, openbmc [-- Attachment #1: Type: text/plain, Size: 7102 bytes --] Hi Ratan, Comments in-line On 2/24/2020 12:07 PM, Ratan Gupta wrote: > > Hi Apparao, > > On 2/20/20 12:49 AM, Puli, Apparao wrote: >> Hi, >> >> I am sorry for late response as this mail is buried under and got >> struck back of my mind. >> >> As i did mentioned in EventService design document, EventLog Monitor >> service is not common across the organizations( Example: Intel uses >> the redfish event logs file and inotify mechanism for monitoring the >> event logs. Where as IBM uses d-bus event log mechanism and they can >> relay on match rules). That said challenges with ResourceType mapping >> will be different in both monitoring mechanisms. This is good point. >> Initially when i started EventService design, i thought we can have >> mapping in bmcweb for ResourceTypes with set of MessageID's for >> Logged events ( As per Intel design) but not sure that may become >> difficult when we expand supported ResourceTypes. > > If I am getting correctly, Here is the flow which Intel uses. > > 1. Individual repo have to push the logs using sd_journal_send which > will write to the file(/var/log/redfish) by using rsyslog daemon > > sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>, > "REDFISH_MESSAGE_ID=%s", > "ResourceEvent.1.0.ResourceCreated",NULL); > > * How you would populate the "OriginOfCondition" during sending > of event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json) > Currently in logServices( logEntry), we are not reporting the "OriginOfCondition" property as per schema. I will check with Jason( Who wrote the logService) and get back on this. BTW can you give me how this information is fetched in IBM way of LogService implementation( D-Bus)? If you already ratified any such, i think we can leverage. If not, We work together for solution. > * Any plan to add resource path in this journal message which > tells that this is the path for which resource created event > generated. > Same as above. > > * Would the path be "Redfish Path/ D-bus Path"? > As per Redfish specification, This should be "@odata.id" which means it should be of resource uri and so we can't use d-bus path here for OriginOfConditions. > > * Where the mapping would be done between D-busPath/Redfish > Resource path? > > > Cons: Every application have to make the change(use sd_journal_send). > My thought is backend application should not be aware of the redfish terminlogy. Having separate process only for this mapping may not be good( No different from maintaining that map inside bmcweb as there won't be any other consumers). Ideal way is, that should be mapped while logging logEntry's itself. But we are not doing it currently which need to be re-looked. Give me some time, I will think and check with other folks and get back. > > *2.* Some application(bmcweb) would do the Inotify on the > path(/var/log/redfish) and send the event once there is any activity > on this file. > > > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As > per Intel design) > > Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type. Initially i thought "ResourceType" based event filtering can be done using minimal mapping( Using MessageID and args). But that will work for minimal set. If the supported ResourceTypes grows, we will have bigger challenges which i can sense it now. Anyway, Supported Resources are completely implementation specific. If this value is empty means, by default all event logs will be sent to subscribers. This is what we can start with before supported ResourceTypes list grows. >> >> As per my reading from below query, You are looking at d-bus match >> rules and ResourceTypes mapping which is more specific to d-bus event >> logging(IBM way of implementing event logging). reading it from >> journal logs will give more information but that will impact the >> performance to large extent. This might be one of the reason why we >> (Intel) uses Redfish message ID while logging redfish events logs to >> journal(You can refer design document for same at >> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). >> In opinion, in your d-bus if you are using some kind of >> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs for >> all events and figure out the way to monitor the journal logs without >> impacting the performance, that should be ok as long as match filters >> are satisfied for Redfish EventService subscriptions and supported >> Types(Again differs with implementation). >> >> Thanks, >> >> -Appu >> >> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote: >>> ApparaRao. >>> >>> As you have shown interest in this feature and submitted the design >>> document, do you have any opinion on this? Do you see any merit in >>> using D-Bus match in bmcweb to create event logs for life cycle >>> events? Please feel free to weigh in. >>> >>> Thanks, >>> Rajes >>> >>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote: >>>> Hi, >>>> >>>> I am going through the bmcweb code for implementing Redfish >>>> EventService based on the design document >>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This >>>> design is hooked to the journal based Redfish Event Logging. For >>>> life cycle events(ResourceAdded, ResourceRemoved, >>>> ResourceUpdated), using D-Bus match, bmcweb can create an event >>>> log. This requires a JSON dictionary, comprising an array of >>>> Redfish Resource Name and the D-Bus path. This approach works only >>>> in case of one to one mapping of Redfish Resource Name and the >>>> D-Bus path. For propertiesChanged events, if the Redfish Resource >>>> property is not on the same D-Bus path or the Redfish Resource >>>> property name is different from the D-Bus property name, then an >>>> additional JSON dictionary to maintain this information is >>>> required. With D-Bus match alone in the bmcweb, Redfish >>>> EventService can't be fully supported. For the Message Registers >>>> and the Resource Types that are supported, the relevant OpenBMC >>>> application must create an event log in the journal using either >>>> the phosphor::logging::entry or sd_journal_send() command. >>>> >>>> After realizing that with D-Bus match in the bmcweb alone can't >>>> help to fully implement EventService, I prefer to avoid using D-Bus >>>> match in bmcweb. Instead, I prefer to modify the OpenBMC >>>> application that generated the event to create an event log in the >>>> journal. Do you see any advantage of using combination of D-Bus >>>> match in the bmcweb wherever it is possible and changes to OpenBMC >>>> application in other cases to create an event log ? >>>> >>>> Your views are highly appreciated. >>>> >>>> Thanks, >>>> Rajes >>> > Thanks > Ratan > > [-- Attachment #2: Type: text/html, Size: 11239 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-02-25 14:06 ` Puli, Apparao @ 2020-05-05 11:43 ` RAJESWARAN THILLAIGOVINDAN 2020-05-26 12:20 ` RAJESWARAN THILLAIGOVINDAN 1 sibling, 0 replies; 44+ messages in thread From: RAJESWARAN THILLAIGOVINDAN @ 2020-05-05 11:43 UTC (permalink / raw) To: Puli, Apparao, Ratan Gupta, openbmc [-- Attachment #1: Type: text/plain, Size: 8540 bytes --] Further to my previous mail, I wanted to see how a JSON dictionary would look like for implementing Redfish EventService based on D-Bus match. The D-Bus match approach is good for implementing life cycle events((ResourceAdded, ResourceRemoved, ResourceUpdated). In bmcweb, a Redfish Resource is viewed as having a set of properties which comes from one or more D-Bus objects. So, when a client subscribes for a ResourceType, using a JSON dictionary, find the D-Bus object(s) mapped to the ResourceType and create matches. When the D-Bus match occurs, again using aJSON dictionary, find the Redfish ResourceType mapped to the D-Bus object and create events. The JSON dictionary should also provide the Redifsh URI which needs to be specified in the event. An example JSON dictionary for mapping the Redifsh ResourceTypes LogEntry and ComputerSystem is available here : https://gist.github.com/trajeswaran/fec230abd36181f85d2f20d09164ec05. In case of LogEntry, there is one to one mapping of ResourceType to D-Bus object. In case of ComputerSystem, the properties comes from multiple D-Bus objects. Do you see any drawback with this approach? Kindly let me know what you think. Thanks in advance. On 25-02-2020 19:36, Puli, Apparao wrote: > > Hi Ratan, > > Comments in-line > > On 2/24/2020 12:07 PM, Ratan Gupta wrote: >> >> Hi Apparao, >> >> On 2/20/20 12:49 AM, Puli, Apparao wrote: >>> Hi, >>> >>> I am sorry for late response as this mail is buried under and got >>> struck back of my mind. >>> >>> As i did mentioned in EventService design document, EventLog Monitor >>> service is not common across the organizations( Example: Intel uses >>> the redfish event logs file and inotify mechanism for monitoring the >>> event logs. Where as IBM uses d-bus event log mechanism and they can >>> relay on match rules). That said challenges with ResourceType >>> mapping will be different in both monitoring mechanisms. This is >>> good point. Initially when i started EventService design, i thought >>> we can have mapping in bmcweb for ResourceTypes with set of >>> MessageID's for Logged events ( As per Intel design) but not sure >>> that may become difficult when we expand supported ResourceTypes. >> >> If I am getting correctly, Here is the flow which Intel uses. >> >> 1. Individual repo have to push the logs using sd_journal_send which >> will write to the file(/var/log/redfish) by using rsyslog daemon >> >> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>, >> "REDFISH_MESSAGE_ID=%s", >> "ResourceEvent.1.0.ResourceCreated",NULL); >> >> * How you would populate the "OriginOfCondition" during sending >> of event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json) >> > Currently in logServices( logEntry), we are not reporting the > "OriginOfCondition" property as per schema. I will check with Jason( > Who wrote the logService) and get back on this. > > BTW can you give me how this information is fetched in IBM way of > LogService implementation( D-Bus)? If you already ratified any such, i > think we can leverage. If not, We work together for solution. > >> * Any plan to add resource path in this journal message which >> tells that this is the path for which resource created event >> generated. >> > Same as above. >> >> * Would the path be "Redfish Path/ D-bus Path"? >> > As per Redfish specification, This should be "@odata.id" which means > it should be of resource uri and so we can't use d-bus path here for > OriginOfConditions. >> >> * Where the mapping would be done between D-busPath/Redfish >> Resource path? >> >> >> Cons: Every application have to make the change(use sd_journal_send). >> My thought is backend application should not be aware of the redfish terminlogy. > > Having separate process only for this mapping may not be good( No > different from maintaining that map inside bmcweb as there won't be > any other consumers). Ideal way is, that should be mapped while > logging logEntry's itself. But we are not doing it currently which > need to be re-looked. Give me some time, I will think and check with > other folks and get back. > >> *2.* Some application(bmcweb) would do the Inotify on the >> path(/var/log/redfish) and send the event once there is any activity >> on this file. >> >> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As >> per Intel design) >> >> Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type. > Initially i thought "ResourceType" based event filtering can be done > using minimal mapping( Using MessageID and args). But that will work > for minimal set. If the supported ResourceTypes grows, we will have > bigger challenges which i can sense it now. Anyway, Supported > Resources are completely implementation specific. If this value is > empty means, by default all event logs will be sent to subscribers. > This is what we can start with before supported ResourceTypes list grows. >>> >>> As per my reading from below query, You are looking at d-bus match >>> rules and ResourceTypes mapping which is more specific to d-bus >>> event logging(IBM way of implementing event logging). reading it >>> from journal logs will give more information but that will impact >>> the performance to large extent. This might be one of the reason why >>> we (Intel) uses Redfish message ID while logging redfish events logs >>> to journal(You can refer design document for same at >>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). >>> In opinion, in your d-bus if you are using some kind of >>> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs for >>> all events and figure out the way to monitor the journal logs >>> without impacting the performance, that should be ok as long as >>> match filters are satisfied for Redfish EventService subscriptions >>> and supported Types(Again differs with implementation). >>> >>> Thanks, >>> >>> -Appu >>> >>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote: >>>> ApparaRao. >>>> >>>> As you have shown interest in this feature and submitted the design >>>> document, do you have any opinion on this? Do you see any merit in >>>> using D-Bus match in bmcweb to create event logs for life cycle >>>> events? Please feel free to weigh in. >>>> >>>> Thanks, >>>> Rajes >>>> >>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote: >>>>> Hi, >>>>> >>>>> I am going through the bmcweb code for implementing Redfish >>>>> EventService based on the design document >>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This >>>>> design is hooked to the journal based Redfish Event Logging. For >>>>> life cycle events(ResourceAdded, ResourceRemoved, >>>>> ResourceUpdated), using D-Bus match, bmcweb can create an event >>>>> log. This requires a JSON dictionary, comprising an array of >>>>> Redfish Resource Name and the D-Bus path. This approach works only >>>>> in case of one to one mapping of Redfish Resource Name and the >>>>> D-Bus path. For propertiesChanged events, if the Redfish Resource >>>>> property is not on the same D-Bus path or the Redfish Resource >>>>> property name is different from the D-Bus property name, then an >>>>> additional JSON dictionary to maintain this information is >>>>> required. With D-Bus match alone in the bmcweb, Redfish >>>>> EventService can't be fully supported. For the Message Registers >>>>> and the Resource Types that are supported, the relevant OpenBMC >>>>> application must create an event log in the journal using either >>>>> the phosphor::logging::entry or sd_journal_send() command. >>>>> >>>>> After realizing that with D-Bus match in the bmcweb alone can't >>>>> help to fully implement EventService, I prefer to avoid using >>>>> D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC >>>>> application that generated the event to create an event log in the >>>>> journal. Do you see any advantage of using combination of D-Bus >>>>> match in the bmcweb wherever it is possible and changes to OpenBMC >>>>> application in other cases to create an event log ? >>>>> >>>>> Your views are highly appreciated. >>>>> >>>>> Thanks, >>>>> Rajes >>>> >> Thanks >> Ratan >> >> [-- Attachment #2: Type: text/html, Size: 13434 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-02-25 14:06 ` Puli, Apparao 2020-05-05 11:43 ` RAJESWARAN THILLAIGOVINDAN @ 2020-05-26 12:20 ` RAJESWARAN THILLAIGOVINDAN 2020-05-27 3:48 ` Puli, Apparao 1 sibling, 1 reply; 44+ messages in thread From: RAJESWARAN THILLAIGOVINDAN @ 2020-05-26 12:20 UTC (permalink / raw) To: Puli, Apparao, Ratan Gupta, openbmc [-- Attachment #1: Type: text/plain, Size: 8212 bytes --] Apparao, I see that you have uploaded a patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) for supporting server sent events. This patch supports event filtering based on registry prefix and/or messageId. I would like to know if you have any plan to support event filtering based on resource type. If so, I would like to work together for a better solution. Earlier, I have proposed a solution based on D-Bus match using a dictionary. With that approach, the major challenge is to map Redfish resource and properties to D-Bus object and properties respectively. If D-Bus applications are modified to include resource type and origin of condition in the event, then there is no need for a map. But,that brings Redfish terminology to the application. Also, this will become an overhead if an OEM is not interested in Redfish event service. Thanks, Rajes On 25-02-2020 19:36, Puli, Apparao wrote: > > Hi Ratan, > > Comments in-line > > On 2/24/2020 12:07 PM, Ratan Gupta wrote: >> >> Hi Apparao, >> >> On 2/20/20 12:49 AM, Puli, Apparao wrote: >>> Hi, >>> >>> I am sorry for late response as this mail is buried under and got >>> struck back of my mind. >>> >>> As i did mentioned in EventService design document, EventLog Monitor >>> service is not common across the organizations( Example: Intel uses >>> the redfish event logs file and inotify mechanism for monitoring the >>> event logs. Where as IBM uses d-bus event log mechanism and they can >>> relay on match rules). That said challenges with ResourceType >>> mapping will be different in both monitoring mechanisms. This is >>> good point. Initially when i started EventService design, i thought >>> we can have mapping in bmcweb for ResourceTypes with set of >>> MessageID's for Logged events ( As per Intel design) but not sure >>> that may become difficult when we expand supported ResourceTypes. >> >> If I am getting correctly, Here is the flow which Intel uses. >> >> 1. Individual repo have to push the logs using sd_journal_send which >> will write to the file(/var/log/redfish) by using rsyslog daemon >> >> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>, >> "REDFISH_MESSAGE_ID=%s", >> "ResourceEvent.1.0.ResourceCreated",NULL); >> >> * How you would populate the "OriginOfCondition" during sending >> of event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json) >> > Currently in logServices( logEntry), we are not reporting the > "OriginOfCondition" property as per schema. I will check with Jason( > Who wrote the logService) and get back on this. > > BTW can you give me how this information is fetched in IBM way of > LogService implementation( D-Bus)? If you already ratified any such, i > think we can leverage. If not, We work together for solution. > >> * Any plan to add resource path in this journal message which >> tells that this is the path for which resource created event >> generated. >> > Same as above. >> >> * Would the path be "Redfish Path/ D-bus Path"? >> > As per Redfish specification, This should be "@odata.id" which means > it should be of resource uri and so we can't use d-bus path here for > OriginOfConditions. >> >> * Where the mapping would be done between D-busPath/Redfish >> Resource path? >> >> >> Cons: Every application have to make the change(use sd_journal_send). >> My thought is backend application should not be aware of the redfish terminlogy. > > Having separate process only for this mapping may not be good( No > different from maintaining that map inside bmcweb as there won't be > any other consumers). Ideal way is, that should be mapped while > logging logEntry's itself. But we are not doing it currently which > need to be re-looked. Give me some time, I will think and check with > other folks and get back. > >> *2.* Some application(bmcweb) would do the Inotify on the >> path(/var/log/redfish) and send the event once there is any activity >> on this file. >> >> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As >> per Intel design) >> >> Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type. > Initially i thought "ResourceType" based event filtering can be done > using minimal mapping( Using MessageID and args). But that will work > for minimal set. If the supported ResourceTypes grows, we will have > bigger challenges which i can sense it now. Anyway, Supported > Resources are completely implementation specific. If this value is > empty means, by default all event logs will be sent to subscribers. > This is what we can start with before supported ResourceTypes list grows. >>> >>> As per my reading from below query, You are looking at d-bus match >>> rules and ResourceTypes mapping which is more specific to d-bus >>> event logging(IBM way of implementing event logging). reading it >>> from journal logs will give more information but that will impact >>> the performance to large extent. This might be one of the reason why >>> we (Intel) uses Redfish message ID while logging redfish events logs >>> to journal(You can refer design document for same at >>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). >>> In opinion, in your d-bus if you are using some kind of >>> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs for >>> all events and figure out the way to monitor the journal logs >>> without impacting the performance, that should be ok as long as >>> match filters are satisfied for Redfish EventService subscriptions >>> and supported Types(Again differs with implementation). >>> >>> Thanks, >>> >>> -Appu >>> >>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote: >>>> ApparaRao. >>>> >>>> As you have shown interest in this feature and submitted the design >>>> document, do you have any opinion on this? Do you see any merit in >>>> using D-Bus match in bmcweb to create event logs for life cycle >>>> events? Please feel free to weigh in. >>>> >>>> Thanks, >>>> Rajes >>>> >>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote: >>>>> Hi, >>>>> >>>>> I am going through the bmcweb code for implementing Redfish >>>>> EventService based on the design document >>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This >>>>> design is hooked to the journal based Redfish Event Logging. For >>>>> life cycle events(ResourceAdded, ResourceRemoved, >>>>> ResourceUpdated), using D-Bus match, bmcweb can create an event >>>>> log. This requires a JSON dictionary, comprising an array of >>>>> Redfish Resource Name and the D-Bus path. This approach works only >>>>> in case of one to one mapping of Redfish Resource Name and the >>>>> D-Bus path. For propertiesChanged events, if the Redfish Resource >>>>> property is not on the same D-Bus path or the Redfish Resource >>>>> property name is different from the D-Bus property name, then an >>>>> additional JSON dictionary to maintain this information is >>>>> required. With D-Bus match alone in the bmcweb, Redfish >>>>> EventService can't be fully supported. For the Message Registers >>>>> and the Resource Types that are supported, the relevant OpenBMC >>>>> application must create an event log in the journal using either >>>>> the phosphor::logging::entry or sd_journal_send() command. >>>>> >>>>> After realizing that with D-Bus match in the bmcweb alone can't >>>>> help to fully implement EventService, I prefer to avoid using >>>>> D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC >>>>> application that generated the event to create an event log in the >>>>> journal. Do you see any advantage of using combination of D-Bus >>>>> match in the bmcweb wherever it is possible and changes to OpenBMC >>>>> application in other cases to create an event log ? >>>>> >>>>> Your views are highly appreciated. >>>>> >>>>> Thanks, >>>>> Rajes >>>> >> Thanks >> Ratan >> >> [-- Attachment #2: Type: text/html, Size: 13084 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-05-26 12:20 ` RAJESWARAN THILLAIGOVINDAN @ 2020-05-27 3:48 ` Puli, Apparao 2020-05-27 11:50 ` RAJESWARAN THILLAIGOVINDAN 0 siblings, 1 reply; 44+ messages in thread From: Puli, Apparao @ 2020-05-27 3:48 UTC (permalink / raw) To: RAJESWARAN THILLAIGOVINDAN, Ratan Gupta, openbmc [-- Attachment #1: Type: text/plain, Size: 9633 bytes --] Hi Rajeswaran, Thanks for your mail. At the moment, I don't have plans to support the "ResourceTypes", "OriginResources" based filtering. Basically Intel uses file systems based redfish event logs( journalctl -> rsync-> filesystem) and doesn't use D-Bus mechanism like IBM uses. So I am not much familiar with D-Bus logging but some of the suggestions: 1) While logging redfish events over D-Bus itself, it can provide details on ResourceTypes and OriginResource URI/Path. This is ideal and most efficient way. Since we walked a walked long distance from start, Its hard to modify all the services to uses these 2 new input parameters while logging events( Requires change in almost all repo's) 2) For resourcesTypes: Can have mapping dictionary against all MessageId's. For OriginResources: I believe, event log over D-Bus is already holding the Path. If so, last 3/4 nodes of uri can be taken and mapped against the resources and that can be used in Event filtering. We did used same mechanism in case of telemetry while mapping MetricReportDefinitions to URI. Hope this helps. Thanks, -Appu On 5/26/2020 5:50 PM, RAJESWARAN THILLAIGOVINDAN wrote: > > Apparao, > > I see that you have uploaded a > patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) for > supporting server sent events. This patch supports event filtering > based on registry prefix and/or messageId. > > I would like to know if you have any plan to support event filtering > based on resource type. If so, I would like to work together for a > better solution. Earlier, I have proposed a solution based on D-Bus > match using a dictionary. With that approach, the major challenge is > to map Redfish resource and properties to D-Bus object and properties > respectively. If D-Bus applications are modified to include resource > type and origin of condition in the event, then there is no need for a > map. But,that brings Redfish terminology to the application. Also, > this will become an overhead if an OEM is not interested in Redfish > event service. > > Thanks, > Rajes > On 25-02-2020 19:36, Puli, Apparao wrote: >> >> Hi Ratan, >> >> Comments in-line >> >> On 2/24/2020 12:07 PM, Ratan Gupta wrote: >>> >>> Hi Apparao, >>> >>> On 2/20/20 12:49 AM, Puli, Apparao wrote: >>>> Hi, >>>> >>>> I am sorry for late response as this mail is buried under and got >>>> struck back of my mind. >>>> >>>> As i did mentioned in EventService design document, EventLog >>>> Monitor service is not common across the organizations( Example: >>>> Intel uses the redfish event logs file and inotify mechanism for >>>> monitoring the event logs. Where as IBM uses d-bus event log >>>> mechanism and they can relay on match rules). That said challenges >>>> with ResourceType mapping will be different in both monitoring >>>> mechanisms. This is good point. Initially when i started >>>> EventService design, i thought we can have mapping in bmcweb for >>>> ResourceTypes with set of MessageID's for Logged events ( As per >>>> Intel design) but not sure that may become difficult when we expand >>>> supported ResourceTypes. >>> >>> If I am getting correctly, Here is the flow which Intel uses. >>> >>> 1. Individual repo have to push the logs using sd_journal_send >>> which will write to the file(/var/log/redfish) by using rsyslog >>> daemon >>> >>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>, >>> "REDFISH_MESSAGE_ID=%s", >>> "ResourceEvent.1.0.ResourceCreated",NULL); >>> >>> * How you would populate the "OriginOfCondition" during >>> sending of event? >>> (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json) >>> >> Currently in logServices( logEntry), we are not reporting the >> "OriginOfCondition" property as per schema. I will check with Jason( >> Who wrote the logService) and get back on this. >> >> BTW can you give me how this information is fetched in IBM way of >> LogService implementation( D-Bus)? If you already ratified any such, >> i think we can leverage. If not, We work together for solution. >> >>> * Any plan to add resource path in this journal message which >>> tells that this is the path for which resource created event >>> generated. >>> >> Same as above. >>> >>> * Would the path be "Redfish Path/ D-bus Path"? >>> >> As per Redfish specification, This should be "@odata.id" which means >> it should be of resource uri and so we can't use d-bus path here for >> OriginOfConditions. >>> >>> * Where the mapping would be done between D-busPath/Redfish >>> Resource path? >>> >>> >>> Cons: Every application have to make the change(use sd_journal_send). >>> My thought is backend application should not be aware of the redfish terminlogy. >> >> Having separate process only for this mapping may not be good( No >> different from maintaining that map inside bmcweb as there won't be >> any other consumers). Ideal way is, that should be mapped while >> logging logEntry's itself. But we are not doing it currently which >> need to be re-looked. Give me some time, I will think and check with >> other folks and get back. >> >>> *2.* Some application(bmcweb) would do the Inotify on the >>> path(/var/log/redfish) and send the event once there is any activity >>> on this file. >>> >>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As >>> per Intel design) >>> >>> Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type. >> Initially i thought "ResourceType" based event filtering can be done >> using minimal mapping( Using MessageID and args). But that will work >> for minimal set. If the supported ResourceTypes grows, we will have >> bigger challenges which i can sense it now. Anyway, Supported >> Resources are completely implementation specific. If this value is >> empty means, by default all event logs will be sent to subscribers. >> This is what we can start with before supported ResourceTypes list >> grows. >>>> >>>> As per my reading from below query, You are looking at d-bus match >>>> rules and ResourceTypes mapping which is more specific to d-bus >>>> event logging(IBM way of implementing event logging). reading it >>>> from journal logs will give more information but that will impact >>>> the performance to large extent. This might be one of the reason >>>> why we (Intel) uses Redfish message ID while logging redfish events >>>> logs to journal(You can refer design document for same at >>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). >>>> In opinion, in your d-bus if you are using some kind of >>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs >>>> for all events and figure out the way to monitor the journal logs >>>> without impacting the performance, that should be ok as long as >>>> match filters are satisfied for Redfish EventService subscriptions >>>> and supported Types(Again differs with implementation). >>>> >>>> Thanks, >>>> >>>> -Appu >>>> >>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote: >>>>> ApparaRao. >>>>> >>>>> As you have shown interest in this feature and submitted the >>>>> design document, do you have any opinion on this? Do you see any >>>>> merit in using D-Bus match in bmcweb to create event logs for life >>>>> cycle events? Please feel free to weigh in. >>>>> >>>>> Thanks, >>>>> Rajes >>>>> >>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote: >>>>>> Hi, >>>>>> >>>>>> I am going through the bmcweb code for implementing Redfish >>>>>> EventService based on the design document >>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This >>>>>> design is hooked to the journal based Redfish Event Logging. For >>>>>> life cycle events(ResourceAdded, ResourceRemoved, >>>>>> ResourceUpdated), using D-Bus match, bmcweb can create an event >>>>>> log. This requires a JSON dictionary, comprising an array of >>>>>> Redfish Resource Name and the D-Bus path. This approach works >>>>>> only in case of one to one mapping of Redfish Resource Name and >>>>>> the D-Bus path. For propertiesChanged events, if the Redfish >>>>>> Resource property is not on the same D-Bus path or the Redfish >>>>>> Resource property name is different from the D-Bus property name, >>>>>> then an additional JSON dictionary to maintain this information >>>>>> is required. With D-Bus match alone in the bmcweb, Redfish >>>>>> EventService can't be fully supported. For the Message Registers >>>>>> and the Resource Types that are supported, the relevant OpenBMC >>>>>> application must create an event log in the journal using either >>>>>> the phosphor::logging::entry or sd_journal_send() command. >>>>>> >>>>>> After realizing that with D-Bus match in the bmcweb alone can't >>>>>> help to fully implement EventService, I prefer to avoid using >>>>>> D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC >>>>>> application that generated the event to create an event log in >>>>>> the journal. Do you see any advantage of using combination of >>>>>> D-Bus match in the bmcweb wherever it is possible and changes to >>>>>> OpenBMC application in other cases to create an event log ? >>>>>> >>>>>> Your views are highly appreciated. >>>>>> >>>>>> Thanks, >>>>>> Rajes >>>>> >>> Thanks >>> Ratan >>> >>> [-- Attachment #2: Type: text/html, Size: 15338 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-05-27 3:48 ` Puli, Apparao @ 2020-05-27 11:50 ` RAJESWARAN THILLAIGOVINDAN 2020-05-27 18:58 ` Puli, Apparao 0 siblings, 1 reply; 44+ messages in thread From: RAJESWARAN THILLAIGOVINDAN @ 2020-05-27 11:50 UTC (permalink / raw) To: Puli, Apparao, Ratan Gupta, openbmc [-- Attachment #1: Type: text/plain, Size: 10624 bytes --] Apparao, Thanks a lot for your suggestions. We lean towards using a dictionary to map Redfish ResourceType to D-Bus objects path and vice versa and then using D-Bus match to generate life cycle events. This way, the changes are limited to bmcweb. The resource type and the origin resource URI will be included in the event log. This requires change in the format of event log file /var/log/redfish. I have commented the same in the server sent event patch that you have uploaded. Kindly see if you can leave the parsing of file to the OEMs. That way, the existing infrastructure can be used by the OEMs to support other filtering mechanisms as defined in the specification. Thanks, Rajes On 27-05-2020 09:18, Puli, Apparao wrote: > > Hi Rajeswaran, > > Thanks for your mail. At the moment, I don't have plans to support > the "ResourceTypes", "OriginResources" based filtering. Basically > Intel uses file systems based redfish event logs( journalctl -> > rsync-> filesystem) and doesn't use D-Bus mechanism like IBM uses. So > I am not much familiar with D-Bus logging but some of the suggestions: > > 1) While logging redfish events over D-Bus itself, it can provide > details on ResourceTypes and OriginResource URI/Path. > > This is ideal and most efficient way. Since we walked a walked > long distance from start, Its hard to modify all the services to uses > these 2 new input parameters while logging events( Requires change in > almost all repo's) > > 2) For resourcesTypes: Can have mapping dictionary against all > MessageId's. For OriginResources: I believe, event log over D-Bus is > already holding the Path. If so, last 3/4 nodes of uri can be taken > and mapped against the resources and that can be used in Event > filtering. We did used same mechanism in case of telemetry while > mapping MetricReportDefinitions to URI. > > Hope this helps. > > Thanks, > > -Appu > > > On 5/26/2020 5:50 PM, RAJESWARAN THILLAIGOVINDAN wrote: >> >> Apparao, >> >> I see that you have uploaded a >> patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) >> for supporting server sent events. This patch supports event >> filtering based on registry prefix and/or messageId. >> >> I would like to know if you have any plan to support event filtering >> based on resource type. If so, I would like to work together for a >> better solution. Earlier, I have proposed a solution based on D-Bus >> match using a dictionary. With that approach, the major challenge is >> to map Redfish resource and properties to D-Bus object and properties >> respectively. If D-Bus applications are modified to include >> resource type and origin of condition in the event, then there is no >> need for a map. But,that brings Redfish terminology to the >> application. Also, this will become an overhead if an OEM is not >> interested in Redfish event service. >> >> Thanks, >> Rajes >> On 25-02-2020 19:36, Puli, Apparao wrote: >>> >>> Hi Ratan, >>> >>> Comments in-line >>> >>> On 2/24/2020 12:07 PM, Ratan Gupta wrote: >>>> >>>> Hi Apparao, >>>> >>>> On 2/20/20 12:49 AM, Puli, Apparao wrote: >>>>> Hi, >>>>> >>>>> I am sorry for late response as this mail is buried under and >>>>> got struck back of my mind. >>>>> >>>>> As i did mentioned in EventService design document, EventLog >>>>> Monitor service is not common across the organizations( Example: >>>>> Intel uses the redfish event logs file and inotify mechanism for >>>>> monitoring the event logs. Where as IBM uses d-bus event log >>>>> mechanism and they can relay on match rules). That said challenges >>>>> with ResourceType mapping will be different in both monitoring >>>>> mechanisms. This is good point. Initially when i started >>>>> EventService design, i thought we can have mapping in bmcweb for >>>>> ResourceTypes with set of MessageID's for Logged events ( As per >>>>> Intel design) but not sure that may become difficult when we >>>>> expand supported ResourceTypes. >>>> >>>> If I am getting correctly, Here is the flow which Intel uses. >>>> >>>> 1. Individual repo have to push the logs using sd_journal_send >>>> which will write to the file(/var/log/redfish) by using rsyslog >>>> daemon >>>> >>>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>, >>>> "REDFISH_MESSAGE_ID=%s", >>>> "ResourceEvent.1.0.ResourceCreated",NULL); >>>> >>>> * How you would populate the "OriginOfCondition" during >>>> sending of event? >>>> (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json) >>>> >>> Currently in logServices( logEntry), we are not reporting the >>> "OriginOfCondition" property as per schema. I will check with Jason( >>> Who wrote the logService) and get back on this. >>> >>> BTW can you give me how this information is fetched in IBM way of >>> LogService implementation( D-Bus)? If you already ratified any such, >>> i think we can leverage. If not, We work together for solution. >>> >>>> * Any plan to add resource path in this journal message which >>>> tells that this is the path for which resource created >>>> event generated. >>>> >>> Same as above. >>>> >>>> * Would the path be "Redfish Path/ D-bus Path"? >>>> >>> As per Redfish specification, This should be "@odata.id" which means >>> it should be of resource uri and so we can't use d-bus path here for >>> OriginOfConditions. >>>> >>>> * Where the mapping would be done between D-busPath/Redfish >>>> Resource path? >>>> >>>> >>>> Cons: Every application have to make the change(use sd_journal_send). >>>> My thought is backend application should not be aware of the redfish terminlogy. >>> >>> Having separate process only for this mapping may not be good( No >>> different from maintaining that map inside bmcweb as there won't be >>> any other consumers). Ideal way is, that should be mapped while >>> logging logEntry's itself. But we are not doing it currently which >>> need to be re-looked. Give me some time, I will think and check with >>> other folks and get back. >>> >>>> *2.* Some application(bmcweb) would do the Inotify on the >>>> path(/var/log/redfish) and send the event once there is any >>>> activity on this file. >>>> >>>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As >>>> per Intel design) >>>> >>>> Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type. >>> Initially i thought "ResourceType" based event filtering can be done >>> using minimal mapping( Using MessageID and args). But that will work >>> for minimal set. If the supported ResourceTypes grows, we will have >>> bigger challenges which i can sense it now. Anyway, Supported >>> Resources are completely implementation specific. If this value is >>> empty means, by default all event logs will be sent to subscribers. >>> This is what we can start with before supported ResourceTypes list >>> grows. >>>>> >>>>> As per my reading from below query, You are looking at d-bus match >>>>> rules and ResourceTypes mapping which is more specific to d-bus >>>>> event logging(IBM way of implementing event logging). reading it >>>>> from journal logs will give more information but that will impact >>>>> the performance to large extent. This might be one of the reason >>>>> why we (Intel) uses Redfish message ID while logging redfish >>>>> events logs to journal(You can refer design document for same at >>>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). >>>>> In opinion, in your d-bus if you are using some kind of >>>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs >>>>> for all events and figure out the way to monitor the journal logs >>>>> without impacting the performance, that should be ok as long as >>>>> match filters are satisfied for Redfish EventService subscriptions >>>>> and supported Types(Again differs with implementation). >>>>> >>>>> Thanks, >>>>> >>>>> -Appu >>>>> >>>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote: >>>>>> ApparaRao. >>>>>> >>>>>> As you have shown interest in this feature and submitted the >>>>>> design document, do you have any opinion on this? Do you see any >>>>>> merit in using D-Bus match in bmcweb to create event logs for >>>>>> life cycle events? Please feel free to weigh in. >>>>>> >>>>>> Thanks, >>>>>> Rajes >>>>>> >>>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I am going through the bmcweb code for implementing Redfish >>>>>>> EventService based on the design document >>>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This >>>>>>> design is hooked to the journal based Redfish Event Logging. For >>>>>>> life cycle events(ResourceAdded, ResourceRemoved, >>>>>>> ResourceUpdated), using D-Bus match, bmcweb can create an event >>>>>>> log. This requires a JSON dictionary, comprising an array of >>>>>>> Redfish Resource Name and the D-Bus path. This approach works >>>>>>> only in case of one to one mapping of Redfish Resource Name and >>>>>>> the D-Bus path. For propertiesChanged events, if the Redfish >>>>>>> Resource property is not on the same D-Bus path or the Redfish >>>>>>> Resource property name is different from the D-Bus property >>>>>>> name, then an additional JSON dictionary to maintain this >>>>>>> information is required. With D-Bus match alone in the bmcweb, >>>>>>> Redfish EventService can't be fully supported. For the Message >>>>>>> Registers and the Resource Types that are supported, the >>>>>>> relevant OpenBMC application must create an event log in the >>>>>>> journal using either the phosphor::logging::entry or >>>>>>> sd_journal_send() command. >>>>>>> >>>>>>> After realizing that with D-Bus match in the bmcweb alone can't >>>>>>> help to fully implement EventService, I prefer to avoid using >>>>>>> D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC >>>>>>> application that generated the event to create an event log in >>>>>>> the journal. Do you see any advantage of using combination of >>>>>>> D-Bus match in the bmcweb wherever it is possible and changes to >>>>>>> OpenBMC application in other cases to create an event log ? >>>>>>> >>>>>>> Your views are highly appreciated. >>>>>>> >>>>>>> Thanks, >>>>>>> Rajes >>>>>> >>>> Thanks >>>> Ratan >>>> >>>> [-- Attachment #2: Type: text/html, Size: 17042 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-05-27 11:50 ` RAJESWARAN THILLAIGOVINDAN @ 2020-05-27 18:58 ` Puli, Apparao 2020-05-28 13:26 ` Ratan Gupta 0 siblings, 1 reply; 44+ messages in thread From: Puli, Apparao @ 2020-05-27 18:58 UTC (permalink / raw) To: RAJESWARAN THILLAIGOVINDAN, Ratan Gupta, openbmc [-- Attachment #1: Type: text/plain, Size: 11603 bytes --] Rajes, The dictionary to map Redfish resourceType and D-Bus object path( I believe URI intern) is good. It will be great if you share design document, if its done. At the moment, redfish event logs file(/var/log/redfish) doesn't have ResourceTypes and OriginResource fields. The Existing redfish event log implementation(log service) also doesn't have support for that. You can propose design change for same along with how it is used in event log service. Same thing, can be adopted to EventService, once its agreed by OEM's( I am thinking, it should go under new OEM specific compiler flag. But that we can ratify later). Thanks, -Appu On 5/27/2020 5:20 PM, RAJESWARAN THILLAIGOVINDAN wrote: > > Apparao, > > Thanks a lot for your suggestions. We lean towards using a dictionary > to map Redfish ResourceType to D-Bus objects path and vice versa and > then using D-Bus match to generate life cycle events. This way, the > changes are limited to bmcweb. The resource type and the origin > resource URI will be included in the event log. This requires change > in the format of event log file /var/log/redfish. I have commented the > same in the server sent event patch that you have uploaded. Kindly see > if you can leave the parsing of file to the OEMs. That way, the > existing infrastructure can be used by the OEMs to support other > filtering mechanisms as defined in the specification. > > Thanks, > Rajes > > > On 27-05-2020 09:18, Puli, Apparao wrote: >> >> Hi Rajeswaran, >> >> Thanks for your mail. At the moment, I don't have plans to support >> the "ResourceTypes", "OriginResources" based filtering. Basically >> Intel uses file systems based redfish event logs( journalctl -> >> rsync-> filesystem) and doesn't use D-Bus mechanism like IBM uses. So >> I am not much familiar with D-Bus logging but some of the suggestions: >> >> 1) While logging redfish events over D-Bus itself, it can provide >> details on ResourceTypes and OriginResource URI/Path. >> >> This is ideal and most efficient way. Since we walked a walked >> long distance from start, Its hard to modify all the services to uses >> these 2 new input parameters while logging events( Requires change in >> almost all repo's) >> >> 2) For resourcesTypes: Can have mapping dictionary against all >> MessageId's. For OriginResources: I believe, event log over D-Bus is >> already holding the Path. If so, last 3/4 nodes of uri can be taken >> and mapped against the resources and that can be used in Event >> filtering. We did used same mechanism in case of telemetry while >> mapping MetricReportDefinitions to URI. >> >> Hope this helps. >> >> Thanks, >> >> -Appu >> >> >> On 5/26/2020 5:50 PM, RAJESWARAN THILLAIGOVINDAN wrote: >>> >>> Apparao, >>> >>> I see that you have uploaded a >>> patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) >>> for supporting server sent events. This patch supports event >>> filtering based on registry prefix and/or messageId. >>> >>> I would like to know if you have any plan to support event filtering >>> based on resource type. If so, I would like to work together for a >>> better solution. Earlier, I have proposed a solution based on D-Bus >>> match using a dictionary. With that approach, the major challenge is >>> to map Redfish resource and properties to D-Bus object and >>> properties respectively. If D-Bus applications are modified to >>> include resource type and origin of condition in the event, then >>> there is no need for a map. But,that brings Redfish terminology to >>> the application. Also, this will become an overhead if an OEM is not >>> interested in Redfish event service. >>> >>> Thanks, >>> Rajes >>> On 25-02-2020 19:36, Puli, Apparao wrote: >>>> >>>> Hi Ratan, >>>> >>>> Comments in-line >>>> >>>> On 2/24/2020 12:07 PM, Ratan Gupta wrote: >>>>> >>>>> Hi Apparao, >>>>> >>>>> On 2/20/20 12:49 AM, Puli, Apparao wrote: >>>>>> Hi, >>>>>> >>>>>> I am sorry for late response as this mail is buried under and >>>>>> got struck back of my mind. >>>>>> >>>>>> As i did mentioned in EventService design document, EventLog >>>>>> Monitor service is not common across the organizations( Example: >>>>>> Intel uses the redfish event logs file and inotify mechanism for >>>>>> monitoring the event logs. Where as IBM uses d-bus event log >>>>>> mechanism and they can relay on match rules). That said >>>>>> challenges with ResourceType mapping will be different in both >>>>>> monitoring mechanisms. This is good point. Initially when i >>>>>> started EventService design, i thought we can have mapping in >>>>>> bmcweb for ResourceTypes with set of MessageID's for Logged >>>>>> events ( As per Intel design) but not sure that may become >>>>>> difficult when we expand supported ResourceTypes. >>>>> >>>>> If I am getting correctly, Here is the flow which Intel uses. >>>>> >>>>> 1. Individual repo have to push the logs using sd_journal_send >>>>> which will write to the file(/var/log/redfish) by using >>>>> rsyslog daemon >>>>> >>>>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>, >>>>> "REDFISH_MESSAGE_ID=%s", >>>>> "ResourceEvent.1.0.ResourceCreated",NULL); >>>>> >>>>> * How you would populate the "OriginOfCondition" during >>>>> sending of event? >>>>> (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json) >>>>> >>>> Currently in logServices( logEntry), we are not reporting the >>>> "OriginOfCondition" property as per schema. I will check with >>>> Jason( Who wrote the logService) and get back on this. >>>> >>>> BTW can you give me how this information is fetched in IBM way of >>>> LogService implementation( D-Bus)? If you already ratified any >>>> such, i think we can leverage. If not, We work together for solution. >>>> >>>>> * Any plan to add resource path in this journal message >>>>> which tells that this is the path for which resource >>>>> created event generated. >>>>> >>>> Same as above. >>>>> >>>>> * Would the path be "Redfish Path/ D-bus Path"? >>>>> >>>> As per Redfish specification, This should be "@odata.id" which >>>> means it should be of resource uri and so we can't use d-bus path >>>> here for OriginOfConditions. >>>>> >>>>> * Where the mapping would be done between D-busPath/Redfish >>>>> Resource path? >>>>> >>>>> >>>>> Cons: Every application have to make the change(use sd_journal_send). >>>>> My thought is backend application should not be aware of the redfish terminlogy. >>>> >>>> Having separate process only for this mapping may not be good( No >>>> different from maintaining that map inside bmcweb as there won't be >>>> any other consumers). Ideal way is, that should be mapped while >>>> logging logEntry's itself. But we are not doing it currently which >>>> need to be re-looked. Give me some time, I will think and check >>>> with other folks and get back. >>>> >>>>> *2.* Some application(bmcweb) would do the Inotify on the >>>>> path(/var/log/redfish) and send the event once there is any >>>>> activity on this file. >>>>> >>>>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As >>>>> per Intel design) >>>>> >>>>> Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type. >>>> Initially i thought "ResourceType" based event filtering can be >>>> done using minimal mapping( Using MessageID and args). But that >>>> will work for minimal set. If the supported ResourceTypes grows, we >>>> will have bigger challenges which i can sense it now. Anyway, >>>> Supported Resources are completely implementation specific. If this >>>> value is empty means, by default all event logs will be sent to >>>> subscribers. This is what we can start with before supported >>>> ResourceTypes list grows. >>>>>> >>>>>> As per my reading from below query, You are looking at d-bus >>>>>> match rules and ResourceTypes mapping which is more specific to >>>>>> d-bus event logging(IBM way of implementing event logging). >>>>>> reading it from journal logs will give more information but that >>>>>> will impact the performance to large extent. This might be one of >>>>>> the reason why we (Intel) uses Redfish message ID while logging >>>>>> redfish events logs to journal(You can refer design document for >>>>>> same at >>>>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). >>>>>> In opinion, in your d-bus if you are using some kind of >>>>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs >>>>>> for all events and figure out the way to monitor the journal logs >>>>>> without impacting the performance, that should be ok as long as >>>>>> match filters are satisfied for Redfish EventService >>>>>> subscriptions and supported Types(Again differs with >>>>>> implementation). >>>>>> >>>>>> Thanks, >>>>>> >>>>>> -Appu >>>>>> >>>>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote: >>>>>>> ApparaRao. >>>>>>> >>>>>>> As you have shown interest in this feature and submitted the >>>>>>> design document, do you have any opinion on this? Do you see any >>>>>>> merit in using D-Bus match in bmcweb to create event logs for >>>>>>> life cycle events? Please feel free to weigh in. >>>>>>> >>>>>>> Thanks, >>>>>>> Rajes >>>>>>> >>>>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I am going through the bmcweb code for implementing Redfish >>>>>>>> EventService based on the design document >>>>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This >>>>>>>> design is hooked to the journal based Redfish Event Logging. >>>>>>>> For life cycle events(ResourceAdded, ResourceRemoved, >>>>>>>> ResourceUpdated), using D-Bus match, bmcweb can create an >>>>>>>> event log. This requires a JSON dictionary, comprising an array >>>>>>>> of Redfish Resource Name and the D-Bus path. This approach >>>>>>>> works only in case of one to one mapping of Redfish Resource >>>>>>>> Name and the D-Bus path. For propertiesChanged events, if the >>>>>>>> Redfish Resource property is not on the same D-Bus path or the >>>>>>>> Redfish Resource property name is different from the D-Bus >>>>>>>> property name, then an additional JSON dictionary to maintain >>>>>>>> this information is required. With D-Bus match alone in the >>>>>>>> bmcweb, Redfish EventService can't be fully supported. For the >>>>>>>> Message Registers and the Resource Types that are supported, >>>>>>>> the relevant OpenBMC application must create an event log in >>>>>>>> the journal using either the phosphor::logging::entry or >>>>>>>> sd_journal_send() command. >>>>>>>> >>>>>>>> After realizing that with D-Bus match in the bmcweb alone can't >>>>>>>> help to fully implement EventService, I prefer to avoid using >>>>>>>> D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC >>>>>>>> application that generated the event to create an event log in >>>>>>>> the journal. Do you see any advantage of using combination of >>>>>>>> D-Bus match in the bmcweb wherever it is possible and changes >>>>>>>> to OpenBMC application in other cases to create an event log ? >>>>>>>> >>>>>>>> Your views are highly appreciated. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Rajes >>>>>>> >>>>> Thanks >>>>> Ratan >>>>> >>>>> [-- Attachment #2: Type: text/html, Size: 18852 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-05-27 18:58 ` Puli, Apparao @ 2020-05-28 13:26 ` Ratan Gupta 2020-05-29 15:45 ` Redfish event log for new local user addition Puli, Apparao 0 siblings, 1 reply; 44+ messages in thread From: Ratan Gupta @ 2020-05-28 13:26 UTC (permalink / raw) To: openbmc, appa >> Puli, Apparao [-- Attachment #1: Type: text/plain, Size: 12160 bytes --] Hi Appu, Can you get me an example say if I want an redfish event to be generated once any local user gets added on the BMC. What would be the steps to be done with the current design and where(which repo)? On 5/28/20 12:28 AM, Puli, Apparao wrote: > > Rajes, > > The dictionary to map Redfish resourceType and D-Bus object path( I > believe URI intern) is good. It will be great if you share design > document, if its done. > > At the moment, redfish event logs file(/var/log/redfish) doesn't have > ResourceTypes and OriginResource fields. The Existing redfish event > log implementation(log service) also doesn't have support for that. > You can propose design change for same along with how it is used in > event log service. Same thing, can be adopted to EventService, once > its agreed by OEM's( I am thinking, it should go under new OEM > specific compiler flag. But that we can ratify later). > > Thanks, > > -Appu > > On 5/27/2020 5:20 PM, RAJESWARAN THILLAIGOVINDAN wrote: >> >> Apparao, >> >> Thanks a lot for your suggestions. We lean towards using a dictionary >> to map Redfish ResourceType to D-Bus objects path and vice versa and >> then using D-Bus match to generate life cycle events. This way, the >> changes are limited to bmcweb. The resource type and the origin >> resource URI will be included in the event log. This requires change >> in the format of event log file /var/log/redfish. I have commented >> the same in the server sent event patch that you have uploaded. >> Kindly see if you can leave the parsing of file to the OEMs. That >> way, the existing infrastructure can be used by the OEMs to support >> other filtering mechanisms as defined in the specification. >> >> Thanks, >> Rajes >> >> >> On 27-05-2020 09:18, Puli, Apparao wrote: >>> >>> Hi Rajeswaran, >>> >>> Thanks for your mail. At the moment, I don't have plans to support >>> the "ResourceTypes", "OriginResources" based filtering. Basically >>> Intel uses file systems based redfish event logs( journalctl -> >>> rsync-> filesystem) and doesn't use D-Bus mechanism like IBM uses. >>> So I am not much familiar with D-Bus logging but some of the >>> suggestions: >>> >>> 1) While logging redfish events over D-Bus itself, it can provide >>> details on ResourceTypes and OriginResource URI/Path. >>> >>> This is ideal and most efficient way. Since we walked a walked >>> long distance from start, Its hard to modify all the services to >>> uses these 2 new input parameters while logging events( Requires >>> change in almost all repo's) >>> >>> 2) For resourcesTypes: Can have mapping dictionary against all >>> MessageId's. For OriginResources: I believe, event log over D-Bus is >>> already holding the Path. If so, last 3/4 nodes of uri can be taken >>> and mapped against the resources and that can be used in Event >>> filtering. We did used same mechanism in case of telemetry while >>> mapping MetricReportDefinitions to URI. >>> >>> Hope this helps. >>> >>> Thanks, >>> >>> -Appu >>> >>> >>> On 5/26/2020 5:50 PM, RAJESWARAN THILLAIGOVINDAN wrote: >>>> >>>> Apparao, >>>> >>>> I see that you have uploaded a >>>> patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) >>>> for supporting server sent events. This patch supports event >>>> filtering based on registry prefix and/or messageId. >>>> >>>> I would like to know if you have any plan to support event >>>> filtering based on resource type. If so, I would like to work >>>> together for a better solution. Earlier, I have proposed a solution >>>> based on D-Bus match using a dictionary. With that approach, the >>>> major challenge is to map Redfish resource and properties to D-Bus >>>> object and properties respectively. If D-Bus applications are >>>> modified to include resource type and origin of condition in the >>>> event, then there is no need for a map. But,that brings Redfish >>>> terminology to the application. Also, this will become an overhead >>>> if an OEM is not interested in Redfish event service. >>>> >>>> Thanks, >>>> Rajes >>>> On 25-02-2020 19:36, Puli, Apparao wrote: >>>>> >>>>> Hi Ratan, >>>>> >>>>> Comments in-line >>>>> >>>>> On 2/24/2020 12:07 PM, Ratan Gupta wrote: >>>>>> >>>>>> Hi Apparao, >>>>>> >>>>>> On 2/20/20 12:49 AM, Puli, Apparao wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I am sorry for late response as this mail is buried under and >>>>>>> got struck back of my mind. >>>>>>> >>>>>>> As i did mentioned in EventService design document, EventLog >>>>>>> Monitor service is not common across the organizations( Example: >>>>>>> Intel uses the redfish event logs file and inotify mechanism for >>>>>>> monitoring the event logs. Where as IBM uses d-bus event log >>>>>>> mechanism and they can relay on match rules). That said >>>>>>> challenges with ResourceType mapping will be different in both >>>>>>> monitoring mechanisms. This is good point. Initially when i >>>>>>> started EventService design, i thought we can have mapping in >>>>>>> bmcweb for ResourceTypes with set of MessageID's for Logged >>>>>>> events ( As per Intel design) but not sure that may become >>>>>>> difficult when we expand supported ResourceTypes. >>>>>> >>>>>> If I am getting correctly, Here is the flow which Intel uses. >>>>>> >>>>>> 1. Individual repo have to push the logs using sd_journal_send >>>>>> which will write to the file(/var/log/redfish) by using >>>>>> rsyslog daemon >>>>>> >>>>>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>, >>>>>> "REDFISH_MESSAGE_ID=%s", >>>>>> "ResourceEvent.1.0.ResourceCreated",NULL); >>>>>> >>>>>> * How you would populate the "OriginOfCondition" during >>>>>> sending of event? >>>>>> (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json) >>>>>> >>>>> Currently in logServices( logEntry), we are not reporting the >>>>> "OriginOfCondition" property as per schema. I will check with >>>>> Jason( Who wrote the logService) and get back on this. >>>>> >>>>> BTW can you give me how this information is fetched in IBM way of >>>>> LogService implementation( D-Bus)? If you already ratified any >>>>> such, i think we can leverage. If not, We work together for >>>>> solution. >>>>> >>>>>> * Any plan to add resource path in this journal message >>>>>> which tells that this is the path for which resource >>>>>> created event generated. >>>>>> >>>>> Same as above. >>>>>> >>>>>> * Would the path be "Redfish Path/ D-bus Path"? >>>>>> >>>>> As per Redfish specification, This should be "@odata.id" which >>>>> means it should be of resource uri and so we can't use d-bus path >>>>> here for OriginOfConditions. >>>>>> >>>>>> * Where the mapping would be done between D-busPath/Redfish >>>>>> Resource path? >>>>>> >>>>>> >>>>>> Cons: Every application have to make the change(use sd_journal_send). >>>>>> My thought is backend application should not be aware of the redfish terminlogy. >>>>> >>>>> Having separate process only for this mapping may not be good( No >>>>> different from maintaining that map inside bmcweb as there won't >>>>> be any other consumers). Ideal way is, that should be mapped while >>>>> logging logEntry's itself. But we are not doing it currently which >>>>> need to be re-looked. Give me some time, I will think and check >>>>> with other folks and get back. >>>>> >>>>>> *2.* Some application(bmcweb) would do the Inotify on the >>>>>> path(/var/log/redfish) and send the event once there is any >>>>>> activity on this file. >>>>>> >>>>>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As >>>>>> per Intel design) >>>>>> >>>>>> Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type. >>>>> Initially i thought "ResourceType" based event filtering can be >>>>> done using minimal mapping( Using MessageID and args). But that >>>>> will work for minimal set. If the supported ResourceTypes grows, >>>>> we will have bigger challenges which i can sense it now. Anyway, >>>>> Supported Resources are completely implementation specific. If >>>>> this value is empty means, by default all event logs will be sent >>>>> to subscribers. This is what we can start with before supported >>>>> ResourceTypes list grows. >>>>>>> >>>>>>> As per my reading from below query, You are looking at d-bus >>>>>>> match rules and ResourceTypes mapping which is more specific to >>>>>>> d-bus event logging(IBM way of implementing event logging). >>>>>>> reading it from journal logs will give more information but that >>>>>>> will impact the performance to large extent. This might be one >>>>>>> of the reason why we (Intel) uses Redfish message ID while >>>>>>> logging redfish events logs to journal(You can refer design >>>>>>> document for same at >>>>>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). >>>>>>> In opinion, in your d-bus if you are using some kind of >>>>>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs >>>>>>> for all events and figure out the way to monitor the journal >>>>>>> logs without impacting the performance, that should be ok as >>>>>>> long as match filters are satisfied for Redfish EventService >>>>>>> subscriptions and supported Types(Again differs with >>>>>>> implementation). >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> -Appu >>>>>>> >>>>>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote: >>>>>>>> ApparaRao. >>>>>>>> >>>>>>>> As you have shown interest in this feature and submitted the >>>>>>>> design document, do you have any opinion on this? Do you see >>>>>>>> any merit in using D-Bus match in bmcweb to create event logs >>>>>>>> for life cycle events? Please feel free to weigh in. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Rajes >>>>>>>> >>>>>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I am going through the bmcweb code for implementing Redfish >>>>>>>>> EventService based on the design document >>>>>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. >>>>>>>>> This design is hooked to the journal based Redfish Event >>>>>>>>> Logging. For life cycle events(ResourceAdded, ResourceRemoved, >>>>>>>>> ResourceUpdated), using D-Bus match, bmcweb can create an >>>>>>>>> event log. This requires a JSON dictionary, comprising an >>>>>>>>> array of Redfish Resource Name and the D-Bus path. This >>>>>>>>> approach works only in case of one to one mapping of Redfish >>>>>>>>> Resource Name and the D-Bus path. For propertiesChanged >>>>>>>>> events, if the Redfish Resource property is not on the same >>>>>>>>> D-Bus path or the Redfish Resource property name is different >>>>>>>>> from the D-Bus property name, then an additional JSON >>>>>>>>> dictionary to maintain this information is required. With >>>>>>>>> D-Bus match alone in the bmcweb, Redfish EventService can't be >>>>>>>>> fully supported. For the Message Registers and the Resource >>>>>>>>> Types that are supported, the relevant OpenBMC application >>>>>>>>> must create an event log in the journal using either the >>>>>>>>> phosphor::logging::entry or sd_journal_send() command. >>>>>>>>> >>>>>>>>> After realizing that with D-Bus match in the bmcweb alone >>>>>>>>> can't help to fully implement EventService, I prefer to avoid >>>>>>>>> using D-Bus match in bmcweb. Instead, I prefer to modify the >>>>>>>>> OpenBMC application that generated the event to create an >>>>>>>>> event log in the journal. Do you see any advantage of using >>>>>>>>> combination of D-Bus match in the bmcweb wherever it is >>>>>>>>> possible and changes to OpenBMC application in other cases to >>>>>>>>> create an event log ? >>>>>>>>> >>>>>>>>> Your views are highly appreciated. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Rajes >>>>>>>> >>>>>> Thanks >>>>>> Ratan >>>>>> >>>>>> Ratan [-- Attachment #2: Type: text/html, Size: 20206 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish event log for new local user addition 2020-05-28 13:26 ` Ratan Gupta @ 2020-05-29 15:45 ` Puli, Apparao 2020-06-02 6:30 ` Ratan Gupta 0 siblings, 1 reply; 44+ messages in thread From: Puli, Apparao @ 2020-05-29 15:45 UTC (permalink / raw) To: Ratan Gupta, openbmc [-- Attachment #1: Type: text/plain, Size: 13233 bytes --] [Subject changed to reflect actual context] Hi Ratan, With current OpenBMC, I don't any event log getting added for new local user creation. To implement that below are two place we need to add support: 1) Define an new message retry ID for user addition(say UserAdded) in bmcweb. https://github.com/openbmc/bmcweb/blob/master/redfish-core/include/registries/openbmc_message_registry.hpp 2) Add a event log during new user addition inside phospor-user-manager(createUser function). https://github.com/openbmc/phosphor-user-manager/blob/master/user_mgr.cpp#L336 While you are adding event log for new local user creation, Please consider other case too( Like Delete, Update, Rename etc..) Thanks, -Appu On 5/28/2020 6:56 PM, Ratan Gupta wrote: > > Hi Appu, > > Can you get me an example say if I want an redfish event to be > generated once any local user gets added on the BMC. > > What would be the steps to be done with the current design and > where(which repo)? > > > On 5/28/20 12:28 AM, Puli, Apparao wrote: >> >> Rajes, >> >> The dictionary to map Redfish resourceType and D-Bus object path( >> I believe URI intern) is good. It will be great if you share design >> document, if its done. >> >> At the moment, redfish event logs file(/var/log/redfish) doesn't have >> ResourceTypes and OriginResource fields. The Existing redfish event >> log implementation(log service) also doesn't have support for that. >> You can propose design change for same along with how it is used in >> event log service. Same thing, can be adopted to EventService, once >> its agreed by OEM's( I am thinking, it should go under new OEM >> specific compiler flag. But that we can ratify later). >> >> Thanks, >> >> -Appu >> >> On 5/27/2020 5:20 PM, RAJESWARAN THILLAIGOVINDAN wrote: >>> >>> Apparao, >>> >>> Thanks a lot for your suggestions. We lean towards using a >>> dictionary to map Redfish ResourceType to D-Bus objects path and >>> vice versa and then using D-Bus match to generate life cycle events. >>> This way, the changes are limited to bmcweb. The resource type and >>> the origin resource URI will be included in the event log. This >>> requires change in the format of event log file /var/log/redfish. I >>> have commented the same in the server sent event patch that you have >>> uploaded. Kindly see if you can leave the parsing of file to the >>> OEMs. That way, the existing infrastructure can be used by the OEMs >>> to support other filtering mechanisms as defined in the specification. >>> >>> Thanks, >>> Rajes >>> >>> >>> On 27-05-2020 09:18, Puli, Apparao wrote: >>>> >>>> Hi Rajeswaran, >>>> >>>> Thanks for your mail. At the moment, I don't have plans to >>>> support the "ResourceTypes", "OriginResources" based filtering. >>>> Basically Intel uses file systems based redfish event logs( >>>> journalctl -> rsync-> filesystem) and doesn't use D-Bus mechanism >>>> like IBM uses. So I am not much familiar with D-Bus logging but >>>> some of the suggestions: >>>> >>>> 1) While logging redfish events over D-Bus itself, it can provide >>>> details on ResourceTypes and OriginResource URI/Path. >>>> >>>> This is ideal and most efficient way. Since we walked a >>>> walked long distance from start, Its hard to modify all the >>>> services to uses these 2 new input parameters while logging events( >>>> Requires change in almost all repo's) >>>> >>>> 2) For resourcesTypes: Can have mapping dictionary against all >>>> MessageId's. For OriginResources: I believe, event log over D-Bus >>>> is already holding the Path. If so, last 3/4 nodes of uri can be >>>> taken and mapped against the resources and that can be used in >>>> Event filtering. We did used same mechanism in case of telemetry >>>> while mapping MetricReportDefinitions to URI. >>>> >>>> Hope this helps. >>>> >>>> Thanks, >>>> >>>> -Appu >>>> >>>> >>>> On 5/26/2020 5:50 PM, RAJESWARAN THILLAIGOVINDAN wrote: >>>>> >>>>> Apparao, >>>>> >>>>> I see that you have uploaded a >>>>> patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) >>>>> for supporting server sent events. This patch supports event >>>>> filtering based on registry prefix and/or messageId. >>>>> >>>>> I would like to know if you have any plan to support event >>>>> filtering based on resource type. If so, I would like to work >>>>> together for a better solution. Earlier, I have proposed a >>>>> solution based on D-Bus match using a dictionary. With that >>>>> approach, the major challenge is to map Redfish resource and >>>>> properties to D-Bus object and properties respectively. If D-Bus >>>>> applications are modified to include resource type and origin of >>>>> condition in the event, then there is no need for a map. But,that >>>>> brings Redfish terminology to the application. Also, this will >>>>> become an overhead if an OEM is not interested in Redfish event >>>>> service. >>>>> >>>>> Thanks, >>>>> Rajes >>>>> On 25-02-2020 19:36, Puli, Apparao wrote: >>>>>> >>>>>> Hi Ratan, >>>>>> >>>>>> Comments in-line >>>>>> >>>>>> On 2/24/2020 12:07 PM, Ratan Gupta wrote: >>>>>>> >>>>>>> Hi Apparao, >>>>>>> >>>>>>> On 2/20/20 12:49 AM, Puli, Apparao wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I am sorry for late response as this mail is buried under and >>>>>>>> got struck back of my mind. >>>>>>>> >>>>>>>> As i did mentioned in EventService design document, EventLog >>>>>>>> Monitor service is not common across the organizations( >>>>>>>> Example: Intel uses the redfish event logs file and inotify >>>>>>>> mechanism for monitoring the event logs. Where as IBM uses >>>>>>>> d-bus event log mechanism and they can relay on match rules). >>>>>>>> That said challenges with ResourceType mapping will be >>>>>>>> different in both monitoring mechanisms. This is good point. >>>>>>>> Initially when i started EventService design, i thought we can >>>>>>>> have mapping in bmcweb for ResourceTypes with set of >>>>>>>> MessageID's for Logged events ( As per Intel design) but not >>>>>>>> sure that may become difficult when we expand supported >>>>>>>> ResourceTypes. >>>>>>> >>>>>>> If I am getting correctly, Here is the flow which Intel uses. >>>>>>> >>>>>>> 1. Individual repo have to push the logs using sd_journal_send >>>>>>> which will write to the file(/var/log/redfish) by using >>>>>>> rsyslog daemon >>>>>>> >>>>>>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>, >>>>>>> "REDFISH_MESSAGE_ID=%s", >>>>>>> "ResourceEvent.1.0.ResourceCreated",NULL); >>>>>>> >>>>>>> * How you would populate the "OriginOfCondition" during >>>>>>> sending of event? >>>>>>> (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json) >>>>>>> >>>>>> Currently in logServices( logEntry), we are not reporting the >>>>>> "OriginOfCondition" property as per schema. I will check with >>>>>> Jason( Who wrote the logService) and get back on this. >>>>>> >>>>>> BTW can you give me how this information is fetched in IBM way of >>>>>> LogService implementation( D-Bus)? If you already ratified any >>>>>> such, i think we can leverage. If not, We work together for >>>>>> solution. >>>>>> >>>>>>> * Any plan to add resource path in this journal message >>>>>>> which tells that this is the path for which resource >>>>>>> created event generated. >>>>>>> >>>>>> Same as above. >>>>>>> >>>>>>> * Would the path be "Redfish Path/ D-bus Path"? >>>>>>> >>>>>> As per Redfish specification, This should be "@odata.id" which >>>>>> means it should be of resource uri and so we can't use d-bus path >>>>>> here for OriginOfConditions. >>>>>>> >>>>>>> * Where the mapping would be done between >>>>>>> D-busPath/Redfish Resource path? >>>>>>> >>>>>>> >>>>>>> Cons: Every application have to make the change(use sd_journal_send). >>>>>>> My thought is backend application should not be aware of the redfish terminlogy. >>>>>> >>>>>> Having separate process only for this mapping may not be good( No >>>>>> different from maintaining that map inside bmcweb as there won't >>>>>> be any other consumers). Ideal way is, that should be mapped >>>>>> while logging logEntry's itself. But we are not doing it >>>>>> currently which need to be re-looked. Give me some time, I will >>>>>> think and check with other folks and get back. >>>>>> >>>>>>> *2.* Some application(bmcweb) would do the Inotify on the >>>>>>> path(/var/log/redfish) and send the event once there is any >>>>>>> activity on this file. >>>>>>> >>>>>>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As >>>>>>> per Intel design) >>>>>>> >>>>>>> Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type. >>>>>> Initially i thought "ResourceType" based event filtering can be >>>>>> done using minimal mapping( Using MessageID and args). But that >>>>>> will work for minimal set. If the supported ResourceTypes grows, >>>>>> we will have bigger challenges which i can sense it now. Anyway, >>>>>> Supported Resources are completely implementation specific. If >>>>>> this value is empty means, by default all event logs will be sent >>>>>> to subscribers. This is what we can start with before supported >>>>>> ResourceTypes list grows. >>>>>>>> >>>>>>>> As per my reading from below query, You are looking at d-bus >>>>>>>> match rules and ResourceTypes mapping which is more specific to >>>>>>>> d-bus event logging(IBM way of implementing event logging). >>>>>>>> reading it from journal logs will give more information but >>>>>>>> that will impact the performance to large extent. This might be >>>>>>>> one of the reason why we (Intel) uses Redfish message ID while >>>>>>>> logging redfish events logs to journal(You can refer design >>>>>>>> document for same at >>>>>>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). >>>>>>>> In opinion, in your d-bus if you are using some kind of >>>>>>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal >>>>>>>> logs for all events and figure out the way to monitor the >>>>>>>> journal logs without impacting the performance, that should be >>>>>>>> ok as long as match filters are satisfied for Redfish >>>>>>>> EventService subscriptions and supported Types(Again differs >>>>>>>> with implementation). >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> -Appu >>>>>>>> >>>>>>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote: >>>>>>>>> ApparaRao. >>>>>>>>> >>>>>>>>> As you have shown interest in this feature and submitted the >>>>>>>>> design document, do you have any opinion on this? Do you see >>>>>>>>> any merit in using D-Bus match in bmcweb to create event logs >>>>>>>>> for life cycle events? Please feel free to weigh in. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Rajes >>>>>>>>> >>>>>>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> I am going through the bmcweb code for implementing Redfish >>>>>>>>>> EventService based on the design document >>>>>>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. >>>>>>>>>> This design is hooked to the journal based Redfish Event >>>>>>>>>> Logging. For life cycle events(ResourceAdded, >>>>>>>>>> ResourceRemoved, ResourceUpdated), using D-Bus match, bmcweb >>>>>>>>>> can create an event log. This requires a JSON dictionary, >>>>>>>>>> comprising an array of Redfish Resource Name and the D-Bus >>>>>>>>>> path. This approach works only in case of one to one mapping >>>>>>>>>> of Redfish Resource Name and the D-Bus path. For >>>>>>>>>> propertiesChanged events, if the Redfish Resource property is >>>>>>>>>> not on the same D-Bus path or the Redfish Resource property >>>>>>>>>> name is different from the D-Bus property name, then an >>>>>>>>>> additional JSON dictionary to maintain this information is >>>>>>>>>> required. With D-Bus match alone in the bmcweb, Redfish >>>>>>>>>> EventService can't be fully supported. For the Message >>>>>>>>>> Registers and the Resource Types that are supported, the >>>>>>>>>> relevant OpenBMC application must create an event log in the >>>>>>>>>> journal using either the phosphor::logging::entry or >>>>>>>>>> sd_journal_send() command. >>>>>>>>>> >>>>>>>>>> After realizing that with D-Bus match in the bmcweb alone >>>>>>>>>> can't help to fully implement EventService, I prefer to avoid >>>>>>>>>> using D-Bus match in bmcweb. Instead, I prefer to modify the >>>>>>>>>> OpenBMC application that generated the event to create an >>>>>>>>>> event log in the journal. Do you see any advantage of using >>>>>>>>>> combination of D-Bus match in the bmcweb wherever it is >>>>>>>>>> possible and changes to OpenBMC application in other cases to >>>>>>>>>> create an event log ? >>>>>>>>>> >>>>>>>>>> Your views are highly appreciated. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Rajes >>>>>>>>> >>>>>>> Thanks >>>>>>> Ratan >>>>>>> >>>>>>> > Ratan [-- Attachment #2: Type: text/html, Size: 22528 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish event log for new local user addition 2020-05-29 15:45 ` Redfish event log for new local user addition Puli, Apparao @ 2020-06-02 6:30 ` Ratan Gupta 0 siblings, 0 replies; 44+ messages in thread From: Ratan Gupta @ 2020-06-02 6:30 UTC (permalink / raw) To: Puli, Apparao, openbmc [-- Attachment #1: Type: text/plain, Size: 13972 bytes --] Hi Appu, Thanks for responding. As the Impact would be on all the openbmc repo where we need to add the event log for any properties change or interface added. I feel it is kind of overhead and not realistic as each and every repo need to know about redfish terminology eg : Redfish Path, Redfish property, Redfish Registry Prefix etc. I feel it is a big work. Ratan On 5/29/20 9:15 PM, Puli, Apparao wrote: > > [Subject changed to reflect actual context] > > Hi Ratan, > > With current OpenBMC, I don't any event log getting added for new > local user creation. To implement that below are two place we need to > add support: > > 1) Define an new message retry ID for user addition(say UserAdded) in > bmcweb. > > https://github.com/openbmc/bmcweb/blob/master/redfish-core/include/registries/openbmc_message_registry.hpp > > 2) Add a event log during new user addition inside > phospor-user-manager(createUser function). > > https://github.com/openbmc/phosphor-user-manager/blob/master/user_mgr.cpp#L336 > > > While you are adding event log for new local user creation, Please > consider other case too( Like Delete, Update, Rename etc..) > > Thanks, > > -Appu > > > On 5/28/2020 6:56 PM, Ratan Gupta wrote: >> >> Hi Appu, >> >> Can you get me an example say if I want an redfish event to be >> generated once any local user gets added on the BMC. >> >> What would be the steps to be done with the current design and >> where(which repo)? >> >> >> On 5/28/20 12:28 AM, Puli, Apparao wrote: >>> >>> Rajes, >>> >>> The dictionary to map Redfish resourceType and D-Bus object path( >>> I believe URI intern) is good. It will be great if you share design >>> document, if its done. >>> >>> At the moment, redfish event logs file(/var/log/redfish) doesn't >>> have ResourceTypes and OriginResource fields. The Existing redfish >>> event log implementation(log service) also doesn't have support for >>> that. You can propose design change for same along with how it is >>> used in event log service. Same thing, can be adopted to >>> EventService, once its agreed by OEM's( I am thinking, it should go >>> under new OEM specific compiler flag. But that we can ratify later). >>> >>> Thanks, >>> >>> -Appu >>> >>> On 5/27/2020 5:20 PM, RAJESWARAN THILLAIGOVINDAN wrote: >>>> >>>> Apparao, >>>> >>>> Thanks a lot for your suggestions. We lean towards using a >>>> dictionary to map Redfish ResourceType to D-Bus objects path and >>>> vice versa and then using D-Bus match to generate life cycle >>>> events. This way, the changes are limited to bmcweb. The resource >>>> type and the origin resource URI will be included in the event log. >>>> This requires change in the format of event log file >>>> /var/log/redfish. I have commented the same in the server sent >>>> event patch that you have uploaded. Kindly see if you can leave the >>>> parsing of file to the OEMs. That way, the existing infrastructure >>>> can be used by the OEMs to support other filtering mechanisms as >>>> defined in the specification. >>>> >>>> Thanks, >>>> Rajes >>>> >>>> >>>> On 27-05-2020 09:18, Puli, Apparao wrote: >>>>> >>>>> Hi Rajeswaran, >>>>> >>>>> Thanks for your mail. At the moment, I don't have plans to >>>>> support the "ResourceTypes", "OriginResources" based filtering. >>>>> Basically Intel uses file systems based redfish event logs( >>>>> journalctl -> rsync-> filesystem) and doesn't use D-Bus mechanism >>>>> like IBM uses. So I am not much familiar with D-Bus logging but >>>>> some of the suggestions: >>>>> >>>>> 1) While logging redfish events over D-Bus itself, it can >>>>> provide details on ResourceTypes and OriginResource URI/Path. >>>>> >>>>> This is ideal and most efficient way. Since we walked a >>>>> walked long distance from start, Its hard to modify all the >>>>> services to uses these 2 new input parameters while logging >>>>> events( Requires change in almost all repo's) >>>>> >>>>> 2) For resourcesTypes: Can have mapping dictionary against all >>>>> MessageId's. For OriginResources: I believe, event log over D-Bus >>>>> is already holding the Path. If so, last 3/4 nodes of uri can be >>>>> taken and mapped against the resources and that can be used in >>>>> Event filtering. We did used same mechanism in case of telemetry >>>>> while mapping MetricReportDefinitions to URI. >>>>> >>>>> Hope this helps. >>>>> >>>>> Thanks, >>>>> >>>>> -Appu >>>>> >>>>> >>>>> On 5/26/2020 5:50 PM, RAJESWARAN THILLAIGOVINDAN wrote: >>>>>> >>>>>> Apparao, >>>>>> >>>>>> I see that you have uploaded a >>>>>> patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) >>>>>> for supporting server sent events. This patch supports event >>>>>> filtering based on registry prefix and/or messageId. >>>>>> >>>>>> I would like to know if you have any plan to support event >>>>>> filtering based on resource type. If so, I would like to work >>>>>> together for a better solution. Earlier, I have proposed a >>>>>> solution based on D-Bus match using a dictionary. With that >>>>>> approach, the major challenge is to map Redfish resource and >>>>>> properties to D-Bus object and properties respectively. If >>>>>> D-Bus applications are modified to include resource type and >>>>>> origin of condition in the event, then there is no need for a >>>>>> map. But,that brings Redfish terminology to the application. >>>>>> Also, this will become an overhead if an OEM is not interested in >>>>>> Redfish event service. >>>>>> >>>>>> Thanks, >>>>>> Rajes >>>>>> On 25-02-2020 19:36, Puli, Apparao wrote: >>>>>>> >>>>>>> Hi Ratan, >>>>>>> >>>>>>> Comments in-line >>>>>>> >>>>>>> On 2/24/2020 12:07 PM, Ratan Gupta wrote: >>>>>>>> >>>>>>>> Hi Apparao, >>>>>>>> >>>>>>>> On 2/20/20 12:49 AM, Puli, Apparao wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I am sorry for late response as this mail is buried under >>>>>>>>> and got struck back of my mind. >>>>>>>>> >>>>>>>>> As i did mentioned in EventService design document, EventLog >>>>>>>>> Monitor service is not common across the organizations( >>>>>>>>> Example: Intel uses the redfish event logs file and inotify >>>>>>>>> mechanism for monitoring the event logs. Where as IBM uses >>>>>>>>> d-bus event log mechanism and they can relay on match rules). >>>>>>>>> That said challenges with ResourceType mapping will be >>>>>>>>> different in both monitoring mechanisms. This is good point. >>>>>>>>> Initially when i started EventService design, i thought we can >>>>>>>>> have mapping in bmcweb for ResourceTypes with set of >>>>>>>>> MessageID's for Logged events ( As per Intel design) but not >>>>>>>>> sure that may become difficult when we expand supported >>>>>>>>> ResourceTypes. >>>>>>>> >>>>>>>> If I am getting correctly, Here is the flow which Intel uses. >>>>>>>> >>>>>>>> 1. Individual repo have to push the logs using sd_journal_send >>>>>>>> which will write to the file(/var/log/redfish) by using >>>>>>>> rsyslog daemon >>>>>>>> >>>>>>>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>, >>>>>>>> "REDFISH_MESSAGE_ID=%s", >>>>>>>> "ResourceEvent.1.0.ResourceCreated",NULL); >>>>>>>> >>>>>>>> * How you would populate the "OriginOfCondition" during >>>>>>>> sending of event? >>>>>>>> (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json) >>>>>>>> >>>>>>> Currently in logServices( logEntry), we are not reporting the >>>>>>> "OriginOfCondition" property as per schema. I will check with >>>>>>> Jason( Who wrote the logService) and get back on this. >>>>>>> >>>>>>> BTW can you give me how this information is fetched in IBM way >>>>>>> of LogService implementation( D-Bus)? If you already ratified >>>>>>> any such, i think we can leverage. If not, We work together for >>>>>>> solution. >>>>>>> >>>>>>>> * Any plan to add resource path in this journal message >>>>>>>> which tells that this is the path for which resource >>>>>>>> created event generated. >>>>>>>> >>>>>>> Same as above. >>>>>>>> >>>>>>>> * Would the path be "Redfish Path/ D-bus Path"? >>>>>>>> >>>>>>> As per Redfish specification, This should be "@odata.id" which >>>>>>> means it should be of resource uri and so we can't use d-bus >>>>>>> path here for OriginOfConditions. >>>>>>>> >>>>>>>> * Where the mapping would be done between >>>>>>>> D-busPath/Redfish Resource path? >>>>>>>> >>>>>>>> >>>>>>>> Cons: Every application have to make the change(use sd_journal_send). >>>>>>>> My thought is backend application should not be aware of the redfish terminlogy. >>>>>>> >>>>>>> Having separate process only for this mapping may not be good( >>>>>>> No different from maintaining that map inside bmcweb as there >>>>>>> won't be any other consumers). Ideal way is, that should be >>>>>>> mapped while logging logEntry's itself. But we are not doing it >>>>>>> currently which need to be re-looked. Give me some time, I will >>>>>>> think and check with other folks and get back. >>>>>>> >>>>>>>> *2.* Some application(bmcweb) would do the Inotify on the >>>>>>>> path(/var/log/redfish) and send the event once there is any >>>>>>>> activity on this file. >>>>>>>> >>>>>>>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As >>>>>>>> per Intel design) >>>>>>>> >>>>>>>> Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type. >>>>>>> Initially i thought "ResourceType" based event filtering can be >>>>>>> done using minimal mapping( Using MessageID and args). But that >>>>>>> will work for minimal set. If the supported ResourceTypes grows, >>>>>>> we will have bigger challenges which i can sense it now. Anyway, >>>>>>> Supported Resources are completely implementation specific. If >>>>>>> this value is empty means, by default all event logs will be >>>>>>> sent to subscribers. This is what we can start with before >>>>>>> supported ResourceTypes list grows. >>>>>>>>> >>>>>>>>> As per my reading from below query, You are looking at d-bus >>>>>>>>> match rules and ResourceTypes mapping which is more specific >>>>>>>>> to d-bus event logging(IBM way of implementing event logging). >>>>>>>>> reading it from journal logs will give more information but >>>>>>>>> that will impact the performance to large extent. This might >>>>>>>>> be one of the reason why we (Intel) uses Redfish message ID >>>>>>>>> while logging redfish events logs to journal(You can refer >>>>>>>>> design document for same at >>>>>>>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md). >>>>>>>>> In opinion, in your d-bus if you are using some kind of >>>>>>>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal >>>>>>>>> logs for all events and figure out the way to monitor the >>>>>>>>> journal logs without impacting the performance, that should be >>>>>>>>> ok as long as match filters are satisfied for Redfish >>>>>>>>> EventService subscriptions and supported Types(Again differs >>>>>>>>> with implementation). >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> -Appu >>>>>>>>> >>>>>>>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote: >>>>>>>>>> ApparaRao. >>>>>>>>>> >>>>>>>>>> As you have shown interest in this feature and submitted the >>>>>>>>>> design document, do you have any opinion on this? Do you see >>>>>>>>>> any merit in using D-Bus match in bmcweb to create event logs >>>>>>>>>> for life cycle events? Please feel free to weigh in. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Rajes >>>>>>>>>> >>>>>>>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> I am going through the bmcweb code for implementing Redfish >>>>>>>>>>> EventService based on the design document >>>>>>>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. >>>>>>>>>>> This design is hooked to the journal based Redfish Event >>>>>>>>>>> Logging. For life cycle events(ResourceAdded, >>>>>>>>>>> ResourceRemoved, ResourceUpdated), using D-Bus match, >>>>>>>>>>> bmcweb can create an event log. This requires a JSON >>>>>>>>>>> dictionary, comprising an array of Redfish Resource Name and >>>>>>>>>>> the D-Bus path. This approach works only in case of one to >>>>>>>>>>> one mapping of Redfish Resource Name and the D-Bus path. For >>>>>>>>>>> propertiesChanged events, if the Redfish Resource property >>>>>>>>>>> is not on the same D-Bus path or the Redfish Resource >>>>>>>>>>> property name is different from the D-Bus property name, >>>>>>>>>>> then an additional JSON dictionary to maintain this >>>>>>>>>>> information is required. With D-Bus match alone in the >>>>>>>>>>> bmcweb, Redfish EventService can't be fully supported. For >>>>>>>>>>> the Message Registers and the Resource Types that are >>>>>>>>>>> supported, the relevant OpenBMC application must create an >>>>>>>>>>> event log in the journal using either the >>>>>>>>>>> phosphor::logging::entry or sd_journal_send() command. >>>>>>>>>>> >>>>>>>>>>> After realizing that with D-Bus match in the bmcweb alone >>>>>>>>>>> can't help to fully implement EventService, I prefer to >>>>>>>>>>> avoid using D-Bus match in bmcweb. Instead, I prefer to >>>>>>>>>>> modify the OpenBMC application that generated the event to >>>>>>>>>>> create an event log in the journal. Do you see any advantage >>>>>>>>>>> of using combination of D-Bus match in the bmcweb wherever >>>>>>>>>>> it is possible and changes to OpenBMC application in other >>>>>>>>>>> cases to create an event log ? >>>>>>>>>>> >>>>>>>>>>> Your views are highly appreciated. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Rajes >>>>>>>>>> >>>>>>>> Thanks >>>>>>>> Ratan >>>>>>>> >>>>>>>> >> Ratan [-- Attachment #2: Type: text/html, Size: 24283 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-01-31 20:53 Redfish EventService Implementation RAJESWARAN THILLAIGOVINDAN 2020-02-09 20:22 ` RAJESWARAN THILLAIGOVINDAN @ 2020-06-08 21:08 ` Brad Bishop 2020-06-09 0:58 ` James Feist 1 sibling, 1 reply; 44+ messages in thread From: Brad Bishop @ 2020-06-08 21:08 UTC (permalink / raw) To: RAJESWARAN THILLAIGOVINDAN, openbmc, apparao.puli On Sat, 2020-02-01 at 02:23 +0530, RAJESWARAN THILLAIGOVINDAN wrote: > Hi, > > I am going through the bmcweb code for implementing Redfish > EventService based on the design document > https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design > is hooked to the journal based Redfish Event Logging. Would anyone else be willing to opine on whether or not they think journal based event schemes are what we want going forward for OpenBMC? My feeling is that they are not - as an alternative IPC mechanism don't we end up re-implementing things that DBus already does? Doesn't it require us to raise the same event twice everywhere (Once with DBus, and once in the journal)? What does journal based eventing do that DBus signals don't do? Please poke holes. thx - brad ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-08 21:08 ` Redfish EventService Implementation Brad Bishop @ 2020-06-09 0:58 ` James Feist 2020-06-15 12:50 ` Ratan Gupta 0 siblings, 1 reply; 44+ messages in thread From: James Feist @ 2020-06-09 0:58 UTC (permalink / raw) To: Brad Bishop, RAJESWARAN THILLAIGOVINDAN, openbmc, apparao.puli Cc: Bills, Jason M On 6/8/2020 2:08 PM, Brad Bishop wrote: > On Sat, 2020-02-01 at 02:23 +0530, RAJESWARAN THILLAIGOVINDAN wrote: >> Hi, >> >> I am going through the bmcweb code for implementing Redfish >> EventService based on the design document >> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design >> is hooked to the journal based Redfish Event Logging. > > Would anyone else be willing to opine on whether or not they think > journal based event schemes are what we want going forward for OpenBMC? > > My feeling is that they are not - as an alternative IPC mechanism don't > we end up re-implementing things that DBus already does? Doesn't it > require us to raise the same event twice everywhere (Once with DBus, and > once in the journal)? What does journal based eventing do that DBus > signals don't do? We don't host log events on DBus at all, so there is no duplicate. The journal gives built in persistence and rotating of logs for large number of events. I know when this came up the last time one of the big issues was supporting thousands of logs wouldn't work well on DBus. > > Please poke holes. > > thx - brad > ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-09 0:58 ` James Feist @ 2020-06-15 12:50 ` Ratan Gupta 2020-06-15 21:42 ` James Feist 0 siblings, 1 reply; 44+ messages in thread From: Ratan Gupta @ 2020-06-15 12:50 UTC (permalink / raw) To: James Feist, Brad Bishop, Puli, Apparao, openbmc, apparao.puli Cc: Bills, Jason M Hi James/Apparao/Brad, I am inclining towards having a separate application for Redfish Logs(like: phosphor-sel-logger), This application does the following. 1) Have the mapping info from Redfish resources to Dbus Resources * This is needed as webserver have to provide the event filtering through Resource Type * eg : Redfish Client may ask as the client is interested in "Account" Resource type i.e all the user account related updates should be given to redfish client. which suggest that there should be mapping from Redfish Resource to Dbus Resource 2) Have the reverse mapping from Dbus Resources to Redfish Resources * This is needed to send the Redfish event if there is any changes in the corresponding D-bus resources. eg BMC state change/network change etc. 3) This application monitors the D-bus event and Log the event in the journal like below eg: sd_journal_send("MESSAGE=%s", "Account Modified", "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s", "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>", "REDFISH_RESOURCE_TYPE=ComputerSystem" "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem", "REDFISH_MESSAGE_ARGS=%s", "Off", NULL); 4) rsyslogd will put all these logs from journal into "/var/log/redfish" file. 5) bmcweb/webserver would inotify this file location and on notification it will send redfish event 6) Event filtering would be done at the bmcweb/webserver side. We already have the infrastructure for seq no 4 and seq no 5 and we wanted to leverage this infrastructure. Please let me know if there is any concern with this approach. Ratan On 6/9/20 6:28 AM, James Feist wrote: > On 6/8/2020 2:08 PM, Brad Bishop wrote: >> On Sat, 2020-02-01 at 02:23 +0530, RAJESWARAN THILLAIGOVINDAN wrote: >>> Hi, >>> >>> I am going through the bmcweb code for implementing Redfish >>> EventService based on the design document >>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design >>> is hooked to the journal based Redfish Event Logging. >> >> Would anyone else be willing to opine on whether or not they think >> journal based event schemes are what we want going forward for OpenBMC? >> >> My feeling is that they are not - as an alternative IPC mechanism don't >> we end up re-implementing things that DBus already does? Doesn't it >> require us to raise the same event twice everywhere (Once with DBus, and >> once in the journal)? What does journal based eventing do that DBus >> signals don't do? > > We don't host log events on DBus at all, so there is no duplicate. The > journal gives built in persistence and rotating of logs for large > number of events. I know when this came up the last time one of the > big issues was supporting thousands of logs wouldn't work well on DBus. > > >> >> Please poke holes. >> >> thx - brad >> ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-15 12:50 ` Ratan Gupta @ 2020-06-15 21:42 ` James Feist 2020-06-16 15:24 ` Patrick Williams 0 siblings, 1 reply; 44+ messages in thread From: James Feist @ 2020-06-15 21:42 UTC (permalink / raw) To: Ratan Gupta, Brad Bishop, Puli, Apparao, openbmc; +Cc: Bills, Jason M On 6/15/2020 5:50 AM, Ratan Gupta wrote: > Hi James/Apparao/Brad, > > I am inclining towards having a separate application for Redfish > Logs(like: phosphor-sel-logger), > This application does the following. > > 1) Have the mapping info from Redfish resources to Dbus Resources > * This is needed as webserver have to provide the event filtering > through Resource Type > * eg : Redfish Client may ask as the client is interested in > "Account" Resource type > i.e all the user account related updates should be given to > redfish client. > which suggest that there should be mapping from Redfish > Resource to Dbus Resource > > 2) Have the reverse mapping from Dbus Resources to Redfish Resources > * This is needed to send the Redfish event if there is any changes > in the > corresponding D-bus resources. eg BMC state change/network > change etc. > > 3) This application monitors the D-bus event and Log the event in the > journal like below Why do we need a listener on d-bus if the application creating the event already knows all the information? Why can't it log straight to the journal? > eg: > sd_journal_send("MESSAGE=%s", "Account Modified", > "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s", > "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>", > "REDFISH_RESOURCE_TYPE=ComputerSystem" > "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem", > "REDFISH_MESSAGE_ARGS=%s", "Off", NULL); If we're going to go down the path of re-implementing logging, I think the goal should be to stop logging things in the Journal that are Redfish specific, and instead log them in some generic format that phosphor logging controls the map. Right now we are bifurcated because the dbus-event-logs, SEL, PEL, and Redfish are all using different methods of logging, that play to their own set of rules. Most repos use phosphor-logging, so instead of creating yet another daemon, if we added support to create a 'System Level' or 'External User' log that has predefined dictionaries of required and optional keys, something like phosphor-dbus-interfaces, we might be able to drop some of these transport specific logs, and have the transports based on the same database (the journal). Then each transport could filter these based on journal entry type, and transform them into the correct log for that transport. I think adding more Redfish specifics into the logs hinders those who do not want Redfish in their systems. > > 4) rsyslogd will put all these logs from journal into "/var/log/redfish" > file. > > 5) bmcweb/webserver would inotify this file location and on notification > it will send redfish event > > 6) Event filtering would be done at the bmcweb/webserver side. > I think this will probably be needed if we want one source of truth for logs. > > We already have the infrastructure for seq no 4 and seq no 5 and we > wanted to leverage this infrastructure. > > Please let me know if there is any concern with this approach. > > Ratan > > On 6/9/20 6:28 AM, James Feist wrote: >> On 6/8/2020 2:08 PM, Brad Bishop wrote: >>> On Sat, 2020-02-01 at 02:23 +0530, RAJESWARAN THILLAIGOVINDAN wrote: >>>> Hi, >>>> >>>> I am going through the bmcweb code for implementing Redfish >>>> EventService based on the design document >>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design >>>> is hooked to the journal based Redfish Event Logging. >>> >>> Would anyone else be willing to opine on whether or not they think >>> journal based event schemes are what we want going forward for OpenBMC? >>> >>> My feeling is that they are not - as an alternative IPC mechanism don't >>> we end up re-implementing things that DBus already does? Doesn't it >>> require us to raise the same event twice everywhere (Once with DBus, and >>> once in the journal)? What does journal based eventing do that DBus >>> signals don't do? >> >> We don't host log events on DBus at all, so there is no duplicate. The >> journal gives built in persistence and rotating of logs for large >> number of events. I know when this came up the last time one of the >> big issues was supporting thousands of logs wouldn't work well on DBus. >> >> >>> >>> Please poke holes. >>> >>> thx - brad >>> ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-15 21:42 ` James Feist @ 2020-06-16 15:24 ` Patrick Williams 2020-06-16 16:11 ` James Feist 0 siblings, 1 reply; 44+ messages in thread From: Patrick Williams @ 2020-06-16 15:24 UTC (permalink / raw) To: James Feist Cc: Ratan Gupta, Brad Bishop, Puli, Apparao, openbmc, Bills, Jason M [-- Attachment #1: Type: text/plain, Size: 2365 bytes --] On Mon, Jun 15, 2020 at 02:42:11PM -0700, James Feist wrote: > On 6/15/2020 5:50 AM, Ratan Gupta wrote: > > eg: > > sd_journal_send("MESSAGE=%s", "Account Modified", > > "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s", > > "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>", > > "REDFISH_RESOURCE_TYPE=ComputerSystem" > > "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem", > > "REDFISH_MESSAGE_ARGS=%s", "Off", NULL); > > If we're going to go down the path of re-implementing logging, I think > the goal should be to stop logging things in the Journal that are > Redfish specific, and instead log them in some generic format that > phosphor logging controls the map. Right now we are bifurcated because > the dbus-event-logs, SEL, PEL, and Redfish are all using different > methods of logging, that play to their own set of rules. Absolutely agree with you here. There is zero reason that applications should start logging specially formed messages with REDFISH_* meta-data. We shouldn't have any applications explicitly know about Redfish except the Redfish providers themselves. This is no different from IPMI, PLDM, or any other external interface. The kind of information presented here as being interesting to expose via Redfish is equally as interesting to me to be able to add to one of our 'FFDC dumps', which could be used for security / forensic work. > Most repos use > phosphor-logging, so instead of creating yet another daemon, if we added > support to create a 'System Level' or 'External User' log that has > predefined dictionaries of required and optional keys, something like > phosphor-dbus-interfaces, we might be able to drop some of these > transport specific logs, and have the transports based on the same > database (the journal). Then each transport could filter these based on > journal entry type, and transform them into the correct log for that > transport. I think adding more Redfish specifics into the logs hinders > those who do not want Redfish in their systems. Can't we do this already today by defining a simple errors/metadata file in phosphor-dbus-interfaces and calling 'logging::report<...>' on it? This will create a record on dbus in phosphor-logging. -- Patrick Williams [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-16 15:24 ` Patrick Williams @ 2020-06-16 16:11 ` James Feist 2020-06-17 12:08 ` Ratan Gupta 0 siblings, 1 reply; 44+ messages in thread From: James Feist @ 2020-06-16 16:11 UTC (permalink / raw) To: Patrick Williams Cc: Bills, Jason M, openbmc, Brad Bishop, Puli, Apparao, Ratan Gupta On 6/16/2020 8:24 AM, Patrick Williams wrote: > On Mon, Jun 15, 2020 at 02:42:11PM -0700, James Feist wrote: >> On 6/15/2020 5:50 AM, Ratan Gupta wrote: >>> eg: >>> sd_journal_send("MESSAGE=%s", "Account Modified", >>> "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s", >>> "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>", >>> "REDFISH_RESOURCE_TYPE=ComputerSystem" >>> "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem", >>> "REDFISH_MESSAGE_ARGS=%s", "Off", NULL); >> >> If we're going to go down the path of re-implementing logging, I think >> the goal should be to stop logging things in the Journal that are >> Redfish specific, and instead log them in some generic format that >> phosphor logging controls the map. Right now we are bifurcated because >> the dbus-event-logs, SEL, PEL, and Redfish are all using different >> methods of logging, that play to their own set of rules. > > Absolutely agree with you here. There is zero reason that applications > should start logging specially formed messages with REDFISH_* meta-data. > We shouldn't have any applications explicitly know about Redfish except > the Redfish providers themselves. This is no different from IPMI, PLDM, > or any other external interface. > > The kind of information presented here as being interesting to expose > via Redfish is equally as interesting to me to be able to add to one of > our 'FFDC dumps', which could be used for security / forensic work. > >> Most repos use >> phosphor-logging, so instead of creating yet another daemon, if we added >> support to create a 'System Level' or 'External User' log that has >> predefined dictionaries of required and optional keys, something like >> phosphor-dbus-interfaces, we might be able to drop some of these >> transport specific logs, and have the transports based on the same >> database (the journal). Then each transport could filter these based on >> journal entry type, and transform them into the correct log for that >> transport. I think adding more Redfish specifics into the logs hinders >> those who do not want Redfish in their systems. > > Can't we do this already today by defining a simple errors/metadata file > in phosphor-dbus-interfaces and calling 'logging::report<...>' on it? > This will create a record on dbus in phosphor-logging. > I think the original concern was with supporting on the order of 10,000 log entries, having this on d-bus seemed impractical. Also the free log rotation the journal provides is useful. Now modifying the logging::report<...> to conditionally log to the journal seems realistic. ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-16 16:11 ` James Feist @ 2020-06-17 12:08 ` Ratan Gupta 2020-06-17 17:16 ` James Feist ` (2 more replies) 0 siblings, 3 replies; 44+ messages in thread From: Ratan Gupta @ 2020-06-17 12:08 UTC (permalink / raw) To: James Feist, Patrick Williams Cc: Bills, Jason M, openbmc, Brad Bishop, Puli, Apparao Hi James,Pattrick. Thanks for giving the feedback On 6/16/20 9:41 PM, James Feist wrote: > On 6/16/2020 8:24 AM, Patrick Williams wrote: >> On Mon, Jun 15, 2020 at 02:42:11PM -0700, James Feist wrote: >>> On 6/15/2020 5:50 AM, Ratan Gupta wrote: >>>> eg: >>>> sd_journal_send("MESSAGE=%s", "Account Modified", >>>> "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s", >>>> "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>", >>>> "REDFISH_RESOURCE_TYPE=ComputerSystem" >>>> "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem", >>>> "REDFISH_MESSAGE_ARGS=%s", "Off", NULL); >>> >>> If we're going to go down the path of re-implementing logging, I think >>> the goal should be to stop logging things in the Journal that are >>> Redfish specific, and instead log them in some generic format that >>> phosphor logging controls the map. Right now we are bifurcated because >>> the dbus-event-logs, SEL, PEL, and Redfish are all using different >>> methods of logging, that play to their own set of rules. >> >> Absolutely agree with you here. There is zero reason that applications >> should start logging specially formed messages with REDFISH_* meta-data. >> We shouldn't have any applications explicitly know about Redfish except >> the Redfish providers themselves. This is no different from IPMI, PLDM, >> or any other external interface. >> >> The kind of information presented here as being interesting to expose >> via Redfish is equally as interesting to me to be able to add to one of >> our 'FFDC dumps', which could be used for security / forensic work. >> >>> Most repos use >>> phosphor-logging, so instead of creating yet another daemon, if we >>> added >>> support to create a 'System Level' or 'External User' log that has >>> predefined dictionaries of required and optional keys, something like >>> phosphor-dbus-interfaces, we might be able to drop some of these >>> transport specific logs, and have the transports based on the same >>> database (the journal). Then each transport could filter these based on >>> journal entry type, and transform them into the correct log for that >>> transport. I think adding more Redfish specifics into the logs hinders >>> those who do not want Redfish in their systems. >> >> Can't we do this already today by defining a simple errors/metadata file >> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it? >> This will create a record on dbus in phosphor-logging. >> > I think the original concern was with supporting on the order of > 10,000 log entries, having this on d-bus seemed impractical. Also the > free log rotation the journal provides is useful. Now modifying the > logging::report<...> to conditionally log to the journal seems realistic. My intention was not to re-implement the logging, my intention was to extend/use the existing design which we already have it below. https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md I was trying not to bring the Redfish specific stuff in each individual repo, instead each transport can listen for Dbus events and write to the journal which goes to their app specific file. Your point is valid that we would be polluting the journal if each and every transport start writing as per their needs. ***** As per the Redfish one of the requirement is we need the log for most of the Dbus Property update/interface added as they are mapped to some Redfish Resource and the bmcweb has to send the Resource updated/modified signal to the Redfish client. ****** As we are in agreement that we want to use the journal for persistence and log rotate feature. We have two options: 1) Each transport interface listens for the Dbus signals and write it to their app specific file. 2) Each openbmc repo must use log::report for each D-bus property update/ interface added. To make this happen, we need the following * common message format which can be used by the all other application (SEL,Redfish) etc. * Transport application will read this common message from the journal and use it as per their needs. Option no 2: It is ideal solution but that requires good amount of work as either each and every D-bus repo should write to the journal for any property update/interface added. or Should SDbusplus do this as each and every application uses the sdbusplus framework to manage the dbus objects Please let me know your thoughts. Ratan ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-17 12:08 ` Ratan Gupta @ 2020-06-17 17:16 ` James Feist 2020-06-17 17:19 ` Bills, Jason M 2020-06-17 20:45 ` Patrick Williams 2 siblings, 0 replies; 44+ messages in thread From: James Feist @ 2020-06-17 17:16 UTC (permalink / raw) To: Ratan Gupta, Patrick Williams Cc: Puli, Apparao, Bills, Jason M, Brad Bishop, openbmc > My intention was not to re-implement the logging, my intention was to > extend/use the existing design which we already have it below. > > https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md > > > I was trying not to bring the Redfish specific stuff in each individual > repo, instead each transport can listen for > Dbus events and write to the journal which goes to their app specific file. > > Your point is valid that we would be polluting the journal if each and > every transport start writing as per their needs. > > ***** As per the Redfish one of the requirement is we need the log for > most of the Dbus Property update/interface added as they > are mapped to some Redfish Resource and the bmcweb has to send the > Resource updated/modified signal to the > Redfish client. ****** > > As we are in agreement that we want to use the journal for persistence > and log rotate feature. > > We have two options: > 1) Each transport interface listens for the Dbus signals and write > it to their app specific file. > 2) Each openbmc repo must use log::report for each D-bus property > update/ interface added. I'm assuming you meant each event added right? I don't see any reason that the logs have anything to do with d-bus, except maybe if it seems worth it for phosphor-logging to catch all the reports and log them to the journal. Although it would probably be more efficient (and easier for phased adoption) to just make the log::report call write to the journal as well. > To make this happen, we need the following > * common message format which can be used by the all other > application (SEL,Redfish) etc. Agree. > * Transport application will read this common message from the > journal and use it as per their needs. > > Option no 2: It is ideal solution but that requires good amount of > work as > either each and every D-bus repo should write to the > journal for any property update/interface added. Why every property/interface added? This would make the logs much to large to be useful. And many times the information needed is specific to the log type. For instance a event for inventory discovered might need the type of inventory, the serial number, the model, etc. To get all that information would be multiple interfaces added events combined together. It would make more sense in this aspect for the inventory daemon to take the parts out of say the Asset, and Chassis interfaces along with the path identifier, and create an entry in the log that is use-able. It would be much more difficult and error-prone for the Redfish or SEL daemon to try to piece these multiple events together to form something useful. > or > Should SDbusplus do this as each and every application > uses the sdbusplus framework to manage the dbus objects > > Please let me know your thoughts. > > Ratan > ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-17 12:08 ` Ratan Gupta 2020-06-17 17:16 ` James Feist @ 2020-06-17 17:19 ` Bills, Jason M 2020-06-17 18:30 ` Andrew Geissler 2020-06-17 20:45 ` Patrick Williams 2 siblings, 1 reply; 44+ messages in thread From: Bills, Jason M @ 2020-06-17 17:19 UTC (permalink / raw) To: openbmc On 6/17/2020 5:08 AM, Ratan Gupta wrote: > Hi James,Pattrick. > > Thanks for giving the feedback > > On 6/16/20 9:41 PM, James Feist wrote: >> On 6/16/2020 8:24 AM, Patrick Williams wrote: >>> On Mon, Jun 15, 2020 at 02:42:11PM -0700, James Feist wrote: >>>> On 6/15/2020 5:50 AM, Ratan Gupta wrote: >>>>> eg: >>>>> sd_journal_send("MESSAGE=%s", "Account Modified", >>>>> "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s", >>>>> "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>", >>>>> "REDFISH_RESOURCE_TYPE=ComputerSystem" >>>>> "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem", >>>>> "REDFISH_MESSAGE_ARGS=%s", "Off", NULL); >>>> >>>> If we're going to go down the path of re-implementing logging, I think >>>> the goal should be to stop logging things in the Journal that are >>>> Redfish specific, and instead log them in some generic format that >>>> phosphor logging controls the map. Right now we are bifurcated because >>>> the dbus-event-logs, SEL, PEL, and Redfish are all using different >>>> methods of logging, that play to their own set of rules. >>> >>> Absolutely agree with you here. There is zero reason that applications >>> should start logging specially formed messages with REDFISH_* meta-data. >>> We shouldn't have any applications explicitly know about Redfish except >>> the Redfish providers themselves. This is no different from IPMI, PLDM, >>> or any other external interface. >>> >>> The kind of information presented here as being interesting to expose >>> via Redfish is equally as interesting to me to be able to add to one of >>> our 'FFDC dumps', which could be used for security / forensic work. >>> >>>> Most repos use >>>> phosphor-logging, so instead of creating yet another daemon, if we >>>> added >>>> support to create a 'System Level' or 'External User' log that has >>>> predefined dictionaries of required and optional keys, something like >>>> phosphor-dbus-interfaces, we might be able to drop some of these >>>> transport specific logs, and have the transports based on the same >>>> database (the journal). Then each transport could filter these based on >>>> journal entry type, and transform them into the correct log for that >>>> transport. I think adding more Redfish specifics into the logs hinders >>>> those who do not want Redfish in their systems. >>> >>> Can't we do this already today by defining a simple errors/metadata file >>> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it? >>> This will create a record on dbus in phosphor-logging. >>> >> I think the original concern was with supporting on the order of >> 10,000 log entries, having this on d-bus seemed impractical. Also the >> free log rotation the journal provides is useful. Now modifying the >> logging::report<...> to conditionally log to the journal seems realistic. > My intention was not to re-implement the logging, my intention was to > extend/use the existing design which we already have it below. > > https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md > > > I was trying not to bring the Redfish specific stuff in each individual > repo, instead each transport can listen for > Dbus events and write to the journal which goes to their app specific file. > > Your point is valid that we would be polluting the journal if each and > every transport start writing as per their needs. > > ***** As per the Redfish one of the requirement is we need the log for > most of the Dbus Property update/interface added as they > are mapped to some Redfish Resource and the bmcweb has to send the > Resource updated/modified signal to the > Redfish client. ****** Why do we want to log on D-Bus property updates? This seems like it will be too noisy for the EventLog. > > As we are in agreement that we want to use the journal for persistence > and log rotate feature. > > We have two options: > 1) Each transport interface listens for the Dbus signals and write > it to their app specific file. > 2) Each openbmc repo must use log::report for each D-bus property > update/ interface added. > To make this happen, we need the following > * common message format which can be used by the all other > application (SEL,Redfish) etc. > * Transport application will read this common message from the > journal and use it as per their needs. > > Option no 2: It is ideal solution but that requires good amount of > work as > either each and every D-bus repo should write to the > journal for any property update/interface added. > or > Should SDbusplus do this as each and every application > uses the sdbusplus framework to manage the dbus objects > > Please let me know your thoughts. > > Ratan > ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-17 17:19 ` Bills, Jason M @ 2020-06-17 18:30 ` Andrew Geissler 0 siblings, 0 replies; 44+ messages in thread From: Andrew Geissler @ 2020-06-17 18:30 UTC (permalink / raw) To: Bills, Jason M; +Cc: openbmc > On Jun 17, 2020, at 12:19 PM, Bills, Jason M <jason.m.bills@linux.intel.com> wrote: > > > > On 6/17/2020 5:08 AM, Ratan Gupta wrote: >> Hi James,Pattrick. >> Thanks for giving the feedback >> On 6/16/20 9:41 PM, James Feist wrote: >>> On 6/16/2020 8:24 AM, Patrick Williams wrote: >>>> On Mon, Jun 15, 2020 at 02:42:11PM -0700, James Feist wrote: >>>>> On 6/15/2020 5:50 AM, Ratan Gupta wrote: >>>>>> eg: >>>>>> sd_journal_send("MESSAGE=%s", "Account Modified", >>>>>> "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s", >>>>>> "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>", >>>>>> "REDFISH_RESOURCE_TYPE=ComputerSystem" >>>>>> "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem", >>>>>> "REDFISH_MESSAGE_ARGS=%s", "Off", NULL); >>>>> >>>>> If we're going to go down the path of re-implementing logging, I think >>>>> the goal should be to stop logging things in the Journal that are >>>>> Redfish specific, and instead log them in some generic format that >>>>> phosphor logging controls the map. Right now we are bifurcated because >>>>> the dbus-event-logs, SEL, PEL, and Redfish are all using different >>>>> methods of logging, that play to their own set of rules. >>>> >>>> Absolutely agree with you here. There is zero reason that applications >>>> should start logging specially formed messages with REDFISH_* meta-data. >>>> We shouldn't have any applications explicitly know about Redfish except >>>> the Redfish providers themselves. This is no different from IPMI, PLDM, >>>> or any other external interface. >>>> >>>> The kind of information presented here as being interesting to expose >>>> via Redfish is equally as interesting to me to be able to add to one of >>>> our 'FFDC dumps', which could be used for security / forensic work. >>>> >>>>> Most repos use >>>>> phosphor-logging, so instead of creating yet another daemon, if we added >>>>> support to create a 'System Level' or 'External User' log that has >>>>> predefined dictionaries of required and optional keys, something like >>>>> phosphor-dbus-interfaces, we might be able to drop some of these >>>>> transport specific logs, and have the transports based on the same >>>>> database (the journal). Then each transport could filter these based on >>>>> journal entry type, and transform them into the correct log for that >>>>> transport. I think adding more Redfish specifics into the logs hinders >>>>> those who do not want Redfish in their systems. >>>> >>>> Can't we do this already today by defining a simple errors/metadata file >>>> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it? >>>> This will create a record on dbus in phosphor-logging. >>>> >>> I think the original concern was with supporting on the order of 10,000 log entries, having this on d-bus seemed impractical. Also the free log rotation the journal provides is useful. Now modifying the logging::report<...> to conditionally log to the journal seems realistic. >> My intention was not to re-implement the logging, my intention was to extend/use the existing design which we already have it below. >> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md I was trying not to bring the Redfish specific stuff in each individual repo, instead each transport can listen for >> Dbus events and write to the journal which goes to their app specific file. >> Your point is valid that we would be polluting the journal if each and every transport start writing as per their needs. >> ***** As per the Redfish one of the requirement is we need the log for most of the Dbus Property update/interface added as they >> are mapped to some Redfish Resource and the bmcweb has to send the Resource updated/modified signal to the >> Redfish client. ****** > Why do we want to log on D-Bus property updates? This seems like it will be too noisy for the EventLog. I don’t think it would be all, just the one’s the different transport layers need. So to get my head straight on this, there is a commit in state-manager to log the redfish events to the journal: https://gerrit.openbmc-project.xyz/c/openbmc/phosphor-state-manager/+/33494 With what Ratan proposed above, we have two options? 1) A separate application that listens for the chassis properties to change and logs those events to the journal in the format the transport layer wants 2) Code within phosphor-state-manager that logs a generic event to the journal which then can be post processed by the different transport layer code. > >> As we are in agreement that we want to use the journal for persistence and log rotate feature. >> We have two options: >> 1) Each transport interface listens for the Dbus signals and write it to their app specific file. >> 2) Each openbmc repo must use log::report for each D-bus property update/ interface added. >> To make this happen, we need the following >> * common message format which can be used by the all other application (SEL,Redfish) etc. >> * Transport application will read this common message from the journal and use it as per their needs. >> Option no 2: It is ideal solution but that requires good amount of work as >> either each and every D-bus repo should write to the journal for any property update/interface added. >> or >> Should SDbusplus do this as each and every application uses the sdbusplus framework to manage the dbus objects >> Please let me know your thoughts. >> Ratan ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-17 12:08 ` Ratan Gupta 2020-06-17 17:16 ` James Feist 2020-06-17 17:19 ` Bills, Jason M @ 2020-06-17 20:45 ` Patrick Williams 2020-06-19 13:26 ` Ratan Gupta 2 siblings, 1 reply; 44+ messages in thread From: Patrick Williams @ 2020-06-17 20:45 UTC (permalink / raw) To: Ratan Gupta Cc: James Feist, Bills, Jason M, openbmc, Brad Bishop, Puli, Apparao [-- Attachment #1: Type: text/plain, Size: 3658 bytes --] On Wed, Jun 17, 2020 at 05:38:47PM +0530, Ratan Gupta wrote: > Hi James,Pattrick. > > >> Can't we do this already today by defining a simple errors/metadata file > >> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it? > >> This will create a record on dbus in phosphor-logging. > >> > > I think the original concern was with supporting on the order of > > 10,000 log entries, having this on d-bus seemed impractical. Also the > > free log rotation the journal provides is useful. Now modifying the > > logging::report<...> to conditionally log to the journal seems realistic. > My intention was not to re-implement the logging, my intention was to > extend/use the existing design which we already have it below. > > https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md > > I was trying not to bring the Redfish specific stuff in each individual > repo, instead each transport can listen for > Dbus events and write to the journal which goes to their app specific file. Good. This wasn't clear from the earlier email. Thanks. > As we are in agreement that we want to use the journal for persistence > and log rotate feature. I'm not convinced there is agreement on this. There has been disagreement about even using the journal for phosphor-logging use since the beginning and I suspect there would be less agreement on another application using it as its own IPC mechanism. Just because a hammer can be used to insert a nail into a board doesn't mean you use it insert any pointy thing into a flat thing. [ Just because the journal provides log rotation and persistance doesn't mean you should use it for every feature needing log rotation and persistance. ] > ***** As per the Redfish one of the requirement is we need the log for > most of the Dbus Property update/interface added as they > are mapped to some Redfish Resource and the bmcweb has to send the > Resource updated/modified signal to the > Redfish client. ****** I don't know Redfish well, so bear with me if there is something obvious I'm missing. But, the first part of this "requirement" doesn't seem to follow from the second part of the "requirement" to me. Sending a signal of a property changing to the Redfish clients is straight-forwawrd; Redfish should subscribe to all the appropriate dbus-events. I don't understand how this implies any sort of logging. Where does the logging part of this requirement come from? > We have two options: > 1) Each transport interface listens for the Dbus signals and write > it to their app specific file. > 2) Each openbmc repo must use log::report for each D-bus property > update/ interface added. #2 is absolutely unworkable on the surface to me. log::report is to create a error entry (xyz.openbmc_project.Logging.Entry), which creates a dbus-object, which would cause log::report to be called, which creates a dbus-object, which ... Even if what you meant was something like logging::log<info>, this seems pretty heavy. I'm not sure this is something that can be inserted into sdbusplus, especially for the ASIO-based object servers, because in many cases applications register their own callback. For the sdbus++ generated server bindings we could squeeze it in. But, what you're proposing here is essentially a "journal-as-dbus-monitor". We'd certainly need to make some measurements on how many kB/s worth of journal entries this would create because I suspect we could end up burning out a NAND flash with as many writes as that would induce. -- Patrick Williams [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-17 20:45 ` Patrick Williams @ 2020-06-19 13:26 ` Ratan Gupta 2020-06-19 22:19 ` Bills, Jason M 2020-06-23 7:27 ` Ratan Gupta 0 siblings, 2 replies; 44+ messages in thread From: Ratan Gupta @ 2020-06-19 13:26 UTC (permalink / raw) To: Patrick Williams Cc: Puli, Apparao, Bills, Jason M, Brad Bishop, James Feist, openbmc [-- Attachment #1: Type: text/plain, Size: 5106 bytes --] On 6/18/20 2:15 AM, Patrick Williams wrote: > On Wed, Jun 17, 2020 at 05:38:47PM +0530, Ratan Gupta wrote: >> Hi James,Pattrick. >> >>>> Can't we do this already today by defining a simple errors/metadata file >>>> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it? >>>> This will create a record on dbus in phosphor-logging. >>>> >>> I think the original concern was with supporting on the order of >>> 10,000 log entries, having this on d-bus seemed impractical. Also the >>> free log rotation the journal provides is useful. Now modifying the >>> logging::report<...> to conditionally log to the journal seems realistic. >> My intention was not to re-implement the logging, my intention was to >> extend/use the existing design which we already have it below. >> >> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md >> >> I was trying not to bring the Redfish specific stuff in each individual >> repo, instead each transport can listen for >> Dbus events and write to the journal which goes to their app specific file. > Good. This wasn't clear from the earlier email. Thanks. > > >> As we are in agreement that we want to use the journal for persistence >> and log rotate feature. > I'm not convinced there is agreement on this. There has been > disagreement about even using the journal for phosphor-logging use since > the beginning and I suspect there would be less agreement on another > application using it as its own IPC mechanism. > > Just because a hammer can be used to insert a nail into a board doesn't > mean you use it insert any pointy thing into a flat thing. [ Just > because the journal provides log rotation and persistance doesn't mean > you should use it for every feature needing log rotation and > persistance. ] > > >> ***** As per the Redfish one of the requirement is we need the log for >> most of the Dbus Property update/interface added as they >> are mapped to some Redfish Resource and the bmcweb has to send the >> Resource updated/modified signal to the >> Redfish client. ****** Jaosn: You asked the following query in other thread /*"Why do we want to log on D-Bus property updates? This seems like it will be too noisy for the EventLog*"/ Eg: Client is interested for an event when ever there is any user add/delete or network configuration change or there is a log entry resource gets created,To handle this request the flow would be Redfish Client subscribe for "ResourceType" eg: "EthernetService,AccountService,LogService" with subordinate resources property as truewhich means the Client is looking for updates on the subscribed resources and the subordinates resource, These redfish resources(EthernetInterface, IP address, ManagerAccount, AccountService) would be mapped to some D-bus Resources, hence some application/bmcweb would monitor the Dbus signals on the interested Dbus objects and send the Redfish event to the subscribed client. Apparo: Please correct me if I am missing something. > I don't know Redfish well, so bear with me if there is something obvious > I'm missing. But, the first part of this "requirement" doesn't seem to > follow from the second part of the "requirement" to me. > > Sending a signal of a property changing to the Redfish clients is > straight-forwawrd; Redfish should subscribe to all the appropriate > dbus-events. I don't understand how this implies any sort of logging. > Where does the logging part of this requirement come from? https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749/16/designs/redfish-eventservice.md#474 While I am reading the redfish spec(https://www.dmtf.org/sites/default/files/standards/documents/DSP0266_1.11.0.pdf) , it is not clear that the events need to be persisted.I will ask in the DMTF for the persistence of the events. >> We have two options: >> 1) Each transport interface listens for the Dbus signals and write >> it to their app specific file. >> 2) Each openbmc repo must use log::report for each D-bus property >> update/ interface added. > #2 is absolutely unworkable on the surface to me. log::report is to > create a error entry (xyz.openbmc_project.Logging.Entry), which creates > a dbus-object, which would cause log::report to be called, which creates > a dbus-object, which ... > > Even if what you meant was something like logging::log<info>, this seems > pretty heavy. I'm not sure this is something that can be inserted into > sdbusplus, especially for the ASIO-based object servers, because in many > cases applications register their own callback. For the sdbus++ > generated server bindings we could squeeze it in. But, what you're > proposing here is essentially a "journal-as-dbus-monitor". We'd > certainly need to make some measurements on how many kB/s worth of > journal entries this would create because I suspect we could end up > burning out a NAND flash with as many writes as that would induce. I would respond on the same once my query gets answered from DMTF. If my query gets answered yes then we have to write on flash but let's wait for it, [-- Attachment #2: Type: text/html, Size: 7343 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-19 13:26 ` Ratan Gupta @ 2020-06-19 22:19 ` Bills, Jason M 2020-06-23 7:27 ` Ratan Gupta 1 sibling, 0 replies; 44+ messages in thread From: Bills, Jason M @ 2020-06-19 22:19 UTC (permalink / raw) To: openbmc On 6/19/2020 6:26 AM, Ratan Gupta wrote: > > On 6/18/20 2:15 AM, Patrick Williams wrote: >> On Wed, Jun 17, 2020 at 05:38:47PM +0530, Ratan Gupta wrote: >>> Hi James,Pattrick. >>> >>>>> Can't we do this already today by defining a simple errors/metadata file >>>>> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it? >>>>> This will create a record on dbus in phosphor-logging. >>>>> >>>> I think the original concern was with supporting on the order of >>>> 10,000 log entries, having this on d-bus seemed impractical. Also the >>>> free log rotation the journal provides is useful. Now modifying the >>>> logging::report<...> to conditionally log to the journal seems realistic. >>> My intention was not to re-implement the logging, my intention was to >>> extend/use the existing design which we already have it below. >>> >>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md >>> >>> I was trying not to bring the Redfish specific stuff in each individual >>> repo, instead each transport can listen for >>> Dbus events and write to the journal which goes to their app specific file. >> Good. This wasn't clear from the earlier email. Thanks. >> >> >>> As we are in agreement that we want to use the journal for persistence >>> and log rotate feature. >> I'm not convinced there is agreement on this. There has been >> disagreement about even using the journal for phosphor-logging use since >> the beginning and I suspect there would be less agreement on another >> application using it as its own IPC mechanism. >> >> Just because a hammer can be used to insert a nail into a board doesn't >> mean you use it insert any pointy thing into a flat thing. [ Just >> because the journal provides log rotation and persistance doesn't mean >> you should use it for every feature needing log rotation and >> persistance. ] >> >> >>> ***** As per the Redfish one of the requirement is we need the log for >>> most of the Dbus Property update/interface added as they >>> are mapped to some Redfish Resource and the bmcweb has to send the >>> Resource updated/modified signal to the >>> Redfish client. ****** > > Jaosn: You asked the following query in other thread /*"Why do we want > to log on D-Bus property updates? This seems like it will be too noisy > for the EventLog*"/ > > Eg: Client is interested for an event when ever there is any user > add/delete or network configuration change or there is a log entry > resource gets created,To handle this request the flow would be My understanding is for the first iteration we are only providing notifications for event messages. So, if it isn't in the message registry, then you cannot register for that event? It seems like the full resource change event notification will be a major feature and should probably get a dedicated design. > > Redfish Client subscribe for "ResourceType" eg: > "EthernetService,AccountService,LogService" with subordinate resources > property as truewhich means the Client is looking for updates on the > subscribed resources and the subordinates resource, These redfish > resources(EthernetInterface, IP address, ManagerAccount, AccountService) > would be mapped to some D-bus Resources, hence some application/bmcweb > would monitor the Dbus signals on the interested Dbus objects and send > the Redfish event to the subscribed client. > > Apparo: Please correct me if I am missing something. > >> I don't know Redfish well, so bear with me if there is something obvious >> I'm missing. But, the first part of this "requirement" doesn't seem to >> follow from the second part of the "requirement" to me. >> >> Sending a signal of a property changing to the Redfish clients is >> straight-forwawrd; Redfish should subscribe to all the appropriate >> dbus-events. I don't understand how this implies any sort of logging. >> Where does the logging part of this requirement come from? > https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749/16/designs/redfish-eventservice.md#474 > > While I am reading the redfish > spec(https://www.dmtf.org/sites/default/files/standards/documents/DSP0266_1.11.0.pdf) > , it is not clear that the events need to be persisted.I will ask in the > DMTF for the persistence of the events. > >>> We have two options: >>> 1) Each transport interface listens for the Dbus signals and write >>> it to their app specific file. >>> 2) Each openbmc repo must use log::report for each D-bus property >>> update/ interface added. >> #2 is absolutely unworkable on the surface to me. log::report is to >> create a error entry (xyz.openbmc_project.Logging.Entry), which creates >> a dbus-object, which would cause log::report to be called, which creates >> a dbus-object, which ... >> >> Even if what you meant was something like logging::log<info>, this seems >> pretty heavy. I'm not sure this is something that can be inserted into >> sdbusplus, especially for the ASIO-based object servers, because in many >> cases applications register their own callback. For the sdbus++ >> generated server bindings we could squeeze it in. But, what you're >> proposing here is essentially a "journal-as-dbus-monitor". We'd >> certainly need to make some measurements on how many kB/s worth of >> journal entries this would create because I suspect we could end up >> burning out a NAND flash with as many writes as that would induce. > I would respond on the same once my query gets answered from DMTF. > If my query gets answered yes then we have to write on flash but let's > wait for it, ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-19 13:26 ` Ratan Gupta 2020-06-19 22:19 ` Bills, Jason M @ 2020-06-23 7:27 ` Ratan Gupta 2020-06-24 16:24 ` James Feist 2020-06-25 17:26 ` Brad Bishop 1 sibling, 2 replies; 44+ messages in thread From: Ratan Gupta @ 2020-06-23 7:27 UTC (permalink / raw) To: Patrick Williams Cc: Puli, Apparao, Bills, Jason M, Brad Bishop, James Feist, openbmc [-- Attachment #1: Type: text/plain, Size: 6517 bytes --] On 6/19/20 6:56 PM, Ratan Gupta wrote: > > > On 6/18/20 2:15 AM, Patrick Williams wrote: >> On Wed, Jun 17, 2020 at 05:38:47PM +0530, Ratan Gupta wrote: >>> Hi James,Pattrick. >>> >>>>> Can't we do this already today by defining a simple errors/metadata file >>>>> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it? >>>>> This will create a record on dbus in phosphor-logging. >>>>> >>>> I think the original concern was with supporting on the order of >>>> 10,000 log entries, having this on d-bus seemed impractical. Also the >>>> free log rotation the journal provides is useful. Now modifying the >>>> logging::report<...> to conditionally log to the journal seems realistic. >>> My intention was not to re-implement the logging, my intention was to >>> extend/use the existing design which we already have it below. >>> >>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md >>> >>> I was trying not to bring the Redfish specific stuff in each individual >>> repo, instead each transport can listen for >>> Dbus events and write to the journal which goes to their app specific file. >> Good. This wasn't clear from the earlier email. Thanks. >> >> >>> As we are in agreement that we want to use the journal for persistence >>> and log rotate feature. >> I'm not convinced there is agreement on this. There has been >> disagreement about even using the journal for phosphor-logging use since >> the beginning and I suspect there would be less agreement on another >> application using it as its own IPC mechanism. >> >> Just because a hammer can be used to insert a nail into a board doesn't >> mean you use it insert any pointy thing into a flat thing. [ Just >> because the journal provides log rotation and persistance doesn't mean >> you should use it for every feature needing log rotation and >> persistance. ] >> >> >>> ***** As per the Redfish one of the requirement is we need the log for >>> most of the Dbus Property update/interface added as they >>> are mapped to some Redfish Resource and the bmcweb has to send the >>> Resource updated/modified signal to the >>> Redfish client. ****** > > Jaosn: You asked the following query in other thread /*"Why do we want > to log on D-Bus property updates? This seems like it will be too > noisy for the EventLog*"/ > > Eg: Client is interested for an event when ever there is any user > add/delete or network configuration change or there is a log entry > resource gets created,To handle this request the flow would be > > Redfish Client subscribe for "ResourceType" eg: > "EthernetService,AccountService,LogService" with subordinate > resources property as truewhich means the Client is looking for > updates on the subscribed resources and the subordinates resource, > These redfish resources(EthernetInterface, IP address, ManagerAccount, > AccountService) would be mapped to some D-bus Resources, hence some > application/bmcweb would monitor the Dbus signals on the interested > Dbus objects and send the Redfish event to the subscribed client. > > Apparo: Please correct me if I am missing something. > >> I don't know Redfish well, so bear with me if there is something obvious >> I'm missing. But, the first part of this "requirement" doesn't seem to >> follow from the second part of the "requirement" to me. >> >> Sending a signal of a property changing to the Redfish clients is >> straight-forwawrd; Redfish should subscribe to all the appropriate >> dbus-events. I don't understand how this implies any sort of logging. >> Where does the logging part of this requirement come from? > https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749/16/designs/redfish-eventservice.md#474 > > While I am reading the redfish > spec(https://www.dmtf.org/sites/default/files/standards/documents/DSP0266_1.11.0.pdf) > , it is not clear that the events need to be persisted.I will ask in > the DMTF for the persistence of the events. I have got an update from the DMTF that persistence is not required to be implemented for the events(https://redfish.dmtf.org/schemas/v1/Event.v1_5_0.json), which implies we don't need to log the events in Journal. More info(https://github.com/DMTF/Redfish/issues/3646) Now updating the proposal where redfish client subscribed for Resource Type based events 1) Client would subscribe for the Redfish Resource(eg:ManagerAccount) to receive events like Account add/delete/modify Hence need for mapping from (RedfishResource to Dbus Resource) 2) Have the mapping info from Redfish resources to DBUS Resources (Some JSon file may have this mapping) 2) Have the reverse mapping from DBUS Resources to Redfish Resources * This is needed to send the Redfish event if there is any changes in the corresponding D-bus resources. eg BMC state change/network change etc. 3) bmcweb would monitor the DBUS events 4) Get the Redfish Path from the Mapping(2) and send the Redfish event 5) Bmcweb would buffer N number of events that can be configurable by redfish client. Buffer would get cleaned up in case of bmcweb restart or BMC reset. James, Apparao: Please let me know if there is any more concern with this approach. > >>> We have two options: >>> 1) Each transport interface listens for the Dbus signals and write >>> it to their app specific file. >>> 2) Each openbmc repo must use log::report for each D-bus property >>> update/ interface added. >> #2 is absolutely unworkable on the surface to me. log::report is to >> create a error entry (xyz.openbmc_project.Logging.Entry), which creates >> a dbus-object, which would cause log::report to be called, which creates >> a dbus-object, which ... >> >> Even if what you meant was something like logging::log<info>, this seems >> pretty heavy. I'm not sure this is something that can be inserted into >> sdbusplus, especially for the ASIO-based object servers, because in many >> cases applications register their own callback. For the sdbus++ >> generated server bindings we could squeeze it in. But, what you're >> proposing here is essentially a "journal-as-dbus-monitor". We'd >> certainly need to make some measurements on how many kB/s worth of >> journal entries this would create because I suspect we could end up >> burning out a NAND flash with as many writes as that would induce. > I would respond on the same once my query gets answered from DMTF. > If my query gets answered yes then we have to write on flash but let's > wait for it, [-- Attachment #2: Type: text/html, Size: 9634 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-23 7:27 ` Ratan Gupta @ 2020-06-24 16:24 ` James Feist 2020-06-24 20:39 ` Patrick Williams ` (2 more replies) 2020-06-25 17:26 ` Brad Bishop 1 sibling, 3 replies; 44+ messages in thread From: James Feist @ 2020-06-24 16:24 UTC (permalink / raw) To: Ratan Gupta, Patrick Williams Cc: Puli, Apparao, Bills, Jason M, Brad Bishop, openbmc > I have got an update from the DMTF that persistence is not required to > be implemented for the > events(https://redfish.dmtf.org/schemas/v1/Event.v1_5_0.json), > which implies we don't need to log the events in Journal. > > More info(https://github.com/DMTF/Redfish/issues/3646) > > Now updating the proposal where redfish client subscribed for Resource > Type based events > 1) Client would subscribe for the Redfish Resource(eg:ManagerAccount) to > receive events like Account add/delete/modify > Hence need for mapping from (RedfishResource to Dbus Resource) > > 2) Have the mapping info from Redfish resources to DBUS Resources (Some > JSon file may have this mapping) This assumes the mapping is static, which on many systems it isn't, right? I think this needs to be developed to see what it would be like. > > 2) Have the reverse mapping from DBUS Resources to Redfish Resources > * This is needed to send the Redfish event if there is any changes > in the corresponding D-bus resources. > eg BMC state change/network change etc. > > 3) bmcweb would monitor the DBUS events Which events specifically? It seems like it would be monitoring lots of events in this proposal. Some examples could be useful. > > 4) Get the Redfish Path from the Mapping(2) and send the Redfish event > > 5) Bmcweb would buffer N number of events that can be configurable by > redfish client. Buffer would get cleaned up in case of bmcweb restart or > BMC reset. > Can you please push a patch to the event service design doc with the change to the design you'd like? Some examples in the design would help with understanding the proposal. This thread is quite long and hard to follow for someone not involved, it'd be good to document the proposed changes. https://github.com/openbmc/docs/blob/master/designs/redfish-eventservice.md ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-24 16:24 ` James Feist @ 2020-06-24 20:39 ` Patrick Williams 2020-06-25 13:45 ` Brad Bishop 2020-06-25 15:49 ` Brad Bishop 2 siblings, 0 replies; 44+ messages in thread From: Patrick Williams @ 2020-06-24 20:39 UTC (permalink / raw) To: James Feist Cc: Ratan Gupta, Puli, Apparao, Bills, Jason M, Brad Bishop, openbmc [-- Attachment #1: Type: text/plain, Size: 591 bytes --] On Wed, Jun 24, 2020 at 09:24:03AM -0700, James Feist wrote: > > 2) Have the mapping info from Redfish resources to DBUS Resources (Some > > JSon file may have this mapping) > > This assumes the mapping is static, which on many systems it isn't, > right? I think this needs to be developed to see what it would be like. > Don't we, in effect, have the mapping from Redfish to dbus by nature of the Redfish providers that create their content from dbus objects? Is there any way to reuse this information to create the corresponding Redfish events? -- Patrick Williams [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-24 16:24 ` James Feist 2020-06-24 20:39 ` Patrick Williams @ 2020-06-25 13:45 ` Brad Bishop 2020-06-25 16:42 ` James Feist 2020-06-25 15:49 ` Brad Bishop 2 siblings, 1 reply; 44+ messages in thread From: Brad Bishop @ 2020-06-25 13:45 UTC (permalink / raw) To: James Feist, Ratan Gupta, Patrick Williams Cc: Bills, Jason M, Puli, Apparao, openbmc On Wed, 2020-06-24 at 09:24 -0700, James Feist wrote: > > 3) bmcweb would monitor the DBUS events > > Which events specifically? It seems like it would be monitoring lots > of events in this proposal. Just curious what is the concern around monitoring lots of events? ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-25 13:45 ` Brad Bishop @ 2020-06-25 16:42 ` James Feist 0 siblings, 0 replies; 44+ messages in thread From: James Feist @ 2020-06-25 16:42 UTC (permalink / raw) To: Brad Bishop, Ratan Gupta, Patrick Williams Cc: Bills, Jason M, Puli, Apparao, openbmc On 6/25/2020 6:45 AM, Brad Bishop wrote: > On Wed, 2020-06-24 at 09:24 -0700, James Feist wrote: >>> 3) bmcweb would monitor the DBUS events >> >> Which events specifically? It seems like it would be monitoring lots >> of events in this proposal. > > Just curious what is the concern around monitoring lots of events? > Performance as always, depending on the number of matches needed. Although a POC could prove this not an issue. ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-24 16:24 ` James Feist 2020-06-24 20:39 ` Patrick Williams 2020-06-25 13:45 ` Brad Bishop @ 2020-06-25 15:49 ` Brad Bishop 2020-06-25 16:44 ` James Feist 2 siblings, 1 reply; 44+ messages in thread From: Brad Bishop @ 2020-06-25 15:49 UTC (permalink / raw) To: James Feist, Ratan Gupta, Patrick Williams Cc: Bills, Jason M, Puli, Apparao, openbmc On Wed, 2020-06-24 at 09:24 -0700, James Feist wrote: > This thread is quite long and hard to follow for someone not involved, > it'd be good to document the proposed changes. You are right of course that this thread has become hard to follow. I think careful selection of reply context really would have helped. FWIW I have asked the team here at IBM to hash out designs to some level of consensus here on the list before making design submissions to gerrit because IMO reviews of half baked designs in gerrit are even harder to follow than email threads like this one 🙂 ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-25 15:49 ` Brad Bishop @ 2020-06-25 16:44 ` James Feist 0 siblings, 0 replies; 44+ messages in thread From: James Feist @ 2020-06-25 16:44 UTC (permalink / raw) To: Brad Bishop, Ratan Gupta, Patrick Williams Cc: Bills, Jason M, Puli, Apparao, openbmc On 6/25/2020 8:49 AM, Brad Bishop wrote: > On Wed, 2020-06-24 at 09:24 -0700, James Feist wrote: >> This thread is quite long and hard to follow for someone not involved, >> it'd be good to document the proposed changes. > > You are right of course that this thread has become hard to follow. I > think careful selection of reply context really would have helped. > > FWIW I have asked the team here at IBM to hash out designs to some level > of consensus here on the list before making design submissions to gerrit > because IMO reviews of half baked designs in gerrit are even harder to > follow than email threads like this one 🙂 > That's good. Was just hoping that the 'final design' wasn't this email, as it would be difficult to see we matched a design. ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-23 7:27 ` Ratan Gupta 2020-06-24 16:24 ` James Feist @ 2020-06-25 17:26 ` Brad Bishop 2020-06-25 18:55 ` Bills, Jason M 1 sibling, 1 reply; 44+ messages in thread From: Brad Bishop @ 2020-06-25 17:26 UTC (permalink / raw) To: Ratan Gupta, Patrick Williams Cc: Bills, Jason M, Puli, Apparao, James Feist, openbmc On Tue, 2020-06-23 at 12:57 +0530, Ratan Gupta wrote: > 1) Client would subscribe for the Redfish Resource(eg:ManagerAccount) > to receive events like Account add/delete/modify. Hence need for > mapping from (RedfishResource to Dbus Resource) > > 2) Have the mapping info from Redfish resources to DBUS Resources > (Some JSon file may have this mapping) It probably doesn't make sense to have any json files. The application logic itself does the mapping. > 3) Have the reverse mapping from DBUS Resources to Redfish Resources > This is needed to send the Redfish event if there is any changes in > the corresponding D-bus resources. eg BMC state change/network change > etc. > > 4) bmcweb would monitor the DBUS events > > 5) Get the Redfish Path from the Mapping(3) and send the Redfish event Let me try to restate 3-5. We already have application logic in bmcweb that maps redfish resources to dbus interfaces, triggered by redfish client requests (like a GET or PATCH). The proposal here I think is to implement the reverse - add application logic in bmcweb that maps dbus interfaces to redfish resources, triggered by dbus signals. I think this is more or less what was suggested by Patrick? > Don't we, in effect, have the mapping from Redfish to dbus by nature > of the Redfish providers that create their content from dbus objects? > James, Apparao: Please let me know if there is any more concern with > this approach. Ratan you had 2x #2s in the flow above, which I fixed. Please let me know if I got it wrong. The main concerns with this approach that I have heard are too many signal matches and complexity of the logic implementing the reverse map. I can definitely see how the logic will get complicated given our current dbus data model. One idea floating around to address these is inventing a journal metadata scheme that is management interface agnostic. I understand the motivation behind that - it is much simpler for an application to slide a single journal entry into the journal with all the metadata needed to generate events, than it is for an application to snoop multiple signals off dbus and pull events out of that. But I wonder if inventing a management interface agnostic scheme for adding events to the journal is really just inventing a new data model for how we represent things in a server - e.g. are we just working around our dbus data model? Maybe we should fix it instead, so that it isn't so difficult for applications to use it? With that said I don't know how to do this and I could use more concrete details on which areas of the data model make it hard to consume signals. Does anyone have any ideas or examples? ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-25 17:26 ` Brad Bishop @ 2020-06-25 18:55 ` Bills, Jason M 2020-06-29 8:00 ` Ratan Gupta 2020-06-29 8:07 ` Ratan Gupta 0 siblings, 2 replies; 44+ messages in thread From: Bills, Jason M @ 2020-06-25 18:55 UTC (permalink / raw) To: openbmc On 6/25/2020 10:26 AM, Brad Bishop wrote: > One idea floating around to address these is inventing a journal > metadata scheme that is management interface agnostic. I understand the > motivation behind that - it is much simpler for an application to slide > a single journal entry into the journal with all the metadata needed to > generate events, than it is for an application to snoop multiple signals > off dbus and pull events out of that. > > But I wonder if inventing a management interface agnostic scheme for > adding events to the journal is really just inventing a new data model > for how we represent things in a server - e.g. are we just working > around our dbus data model? Maybe we should fix it instead, so that it > isn't so difficult for applications to use it? With that said I don't > know how to do this and I could use more concrete details on which areas > of the data model make it hard to consume signals. Does anyone have any > ideas or examples? > On this, I think we may want to separate logging vs. eventing both in this feature discussion and in the tools we want to use. When we were talking about logging, I think the journal made sense since it is designed for logging and has benefits around that usage. However, I agree that it doesn't seem like the right tool for sending and receiving events and signals and that D-Bus sounds like the right tool for that. I think I'm still a little confused at the scope. My understanding was that this initial design for EventService was only for monitoring event messages and not resources in general. It seems like it may not make sense to try to use the same tools and approach for both message monitoring and resource monitoring? Do we need to treat them separately for now to simplify the discussion? ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-25 18:55 ` Bills, Jason M @ 2020-06-29 8:00 ` Ratan Gupta 2020-06-29 8:07 ` Ratan Gupta 1 sibling, 0 replies; 44+ messages in thread From: Ratan Gupta @ 2020-06-29 8:00 UTC (permalink / raw) To: Bills, Jason M, openbmc Hi Jason,james On 6/26/20 12:25 AM, Bills, Jason M wrote: > > > On 6/25/2020 10:26 AM, Brad Bishop wrote: >> One idea floating around to address these is inventing a journal >> metadata scheme that is management interface agnostic. I understand the >> motivation behind that - it is much simpler for an application to slide >> a single journal entry into the journal with all the metadata needed to >> generate events, than it is for an application to snoop multiple signals >> off dbus and pull events out of that. >> >> But I wonder if inventing a management interface agnostic scheme for >> adding events to the journal is really just inventing a new data model >> for how we represent things in a server - e.g. are we just working >> around our dbus data model? Maybe we should fix it instead, so that it >> isn't so difficult for applications to use it? With that said I don't >> know how to do this and I could use more concrete details on which areas >> of the data model make it hard to consume signals. Does anyone have any >> ideas or examples? >> > > On this, I think we may want to separate logging vs. eventing both in > this feature discussion and in the tools we want to use. > > When we were talking about logging, I think the journal made sense > since it is designed for logging and has benefits around that usage. > However, I agree that it doesn't seem like the right tool for sending > and receiving events and signals and that D-Bus sounds like the right > tool for that. > > I think I'm still a little confused at the scope. My understanding > was that this initial design for EventService was only for monitoring > event messages and not resources in general. It seems like it may not > make sense to try to use the same tools and approach for both message > monitoring and resource monitoring? Do we need to treat them > separately for now to simplify the discussion? Jason, When you say event messages? What do you mean, Do you mean to say "/redfish/v1/Systems/system/Logservices/eventlog"? If yes then this should also go as Resource Event, When ever any log entry gets created under System Log (/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would notify to the Redfish client saying that "ResourceCreated" with the URL of the Resource. After recieving this event Redfish client will do a GET request on the URL(retrieved as part of event) to get the content of the log. I would be coming up with few design approaches and downside with each approach to take it to conclusion. Ratan ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-25 18:55 ` Bills, Jason M 2020-06-29 8:00 ` Ratan Gupta @ 2020-06-29 8:07 ` Ratan Gupta 2020-06-29 15:33 ` Bills, Jason M 1 sibling, 1 reply; 44+ messages in thread From: Ratan Gupta @ 2020-06-29 8:07 UTC (permalink / raw) To: Bills, Jason M, openbmc On 6/26/20 12:25 AM, Bills, Jason M wrote: > > > On 6/25/2020 10:26 AM, Brad Bishop wrote: >> One idea floating around to address these is inventing a journal >> metadata scheme that is management interface agnostic. I understand the >> motivation behind that - it is much simpler for an application to slide >> a single journal entry into the journal with all the metadata needed to >> generate events, than it is for an application to snoop multiple signals >> off dbus and pull events out of that. >> >> But I wonder if inventing a management interface agnostic scheme for >> adding events to the journal is really just inventing a new data model >> for how we represent things in a server - e.g. are we just working >> around our dbus data model? Maybe we should fix it instead, so that it >> isn't so difficult for applications to use it? With that said I don't >> know how to do this and I could use more concrete details on which areas >> of the data model make it hard to consume signals. Does anyone have any >> ideas or examples? >> > > On this, I think we may want to separate logging vs. eventing both in > this feature discussion and in the tools we want to use. > > When we were talking about logging, I think the journal made sense > since it is designed for logging and has benefits around that usage. > However, I agree that it doesn't seem like the right tool for sending > and receiving events and signals and that D-Bus sounds like the right > tool for that. > > I think I'm still a little confused at the scope. My understanding > was that this initial design for EventService was only for monitoring > event messages and not resources in general. It seems like it may not > make sense to try to use the same tools and approach for both message > monitoring and resource monitoring? Do we need to treat them > separately for now to simplify the discussion? Jason, When you say event messages? What do you mean, Do you mean to say "/redfish/v1/Systems/system/Logservices/eventlog"? If yes then this should also go as Resource Event, When ever any log entry gets created under System Log (/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would notify to the Redfish client saying that "ResourceCreated" with the URL of the Resource. After receiving this event Redfish client will do a GET request on the URL(retrieved as part of event) to get the content of the log. This will become generic infra for all types of events. I would be coming up with few design approaches and downside with each approach to take it to conclusion. Ratan ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-29 8:07 ` Ratan Gupta @ 2020-06-29 15:33 ` Bills, Jason M 2020-07-03 10:15 ` Ratan Gupta 0 siblings, 1 reply; 44+ messages in thread From: Bills, Jason M @ 2020-06-29 15:33 UTC (permalink / raw) To: openbmc On 6/29/2020 1:07 AM, Ratan Gupta wrote: > > On 6/26/20 12:25 AM, Bills, Jason M wrote: >> >> >> On 6/25/2020 10:26 AM, Brad Bishop wrote: >>> One idea floating around to address these is inventing a journal >>> metadata scheme that is management interface agnostic. I understand the >>> motivation behind that - it is much simpler for an application to slide >>> a single journal entry into the journal with all the metadata needed to >>> generate events, than it is for an application to snoop multiple signals >>> off dbus and pull events out of that. >>> >>> But I wonder if inventing a management interface agnostic scheme for >>> adding events to the journal is really just inventing a new data model >>> for how we represent things in a server - e.g. are we just working >>> around our dbus data model? Maybe we should fix it instead, so that it >>> isn't so difficult for applications to use it? With that said I don't >>> know how to do this and I could use more concrete details on which areas >>> of the data model make it hard to consume signals. Does anyone have any >>> ideas or examples? >>> >> >> On this, I think we may want to separate logging vs. eventing both in >> this feature discussion and in the tools we want to use. >> >> When we were talking about logging, I think the journal made sense >> since it is designed for logging and has benefits around that usage. >> However, I agree that it doesn't seem like the right tool for sending >> and receiving events and signals and that D-Bus sounds like the right >> tool for that. >> >> I think I'm still a little confused at the scope. My understanding >> was that this initial design for EventService was only for monitoring >> event messages and not resources in general. It seems like it may not >> make sense to try to use the same tools and approach for both message >> monitoring and resource monitoring? Do we need to treat them >> separately for now to simplify the discussion? > Jason, When you say event messages? What do you mean, Do you mean to say > "/redfish/v1/Systems/system/Logservices/eventlog"? > > If yes then this should also go as Resource Event, When ever any log > entry gets created under System Log > (/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would > notify to the Redfish client saying that "ResourceCreated" with the URL > of the Resource. Yes, new entries under "/redfish/v1/Systems/system/Logservices/eventlog", but I thought you could register for specific MessageIDs, so it's not just a generic "new resource" event like others would be. > > After receiving this event Redfish client will do a GET request on the > URL(retrieved as part of event) to get the content of the log. > > This will become generic infra for all types of events. What I'm saying is I don't know if there is a good generic solution to cover both the EventLog and all other resources. I believe the current EventService implementation was designed only for EventLog and may not work well for generic resource events. > > I would be coming up with few design approaches and downside with each > approach to take it to conclusion. Thanks! What I'm proposing is that we clarify or possibly separate the discussions about EventLog vs. generic resources to avoid confusion and come up with the right solutions for each. > > Ratan > ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-06-29 15:33 ` Bills, Jason M @ 2020-07-03 10:15 ` Ratan Gupta 2020-07-06 21:30 ` Bills, Jason M 0 siblings, 1 reply; 44+ messages in thread From: Ratan Gupta @ 2020-07-03 10:15 UTC (permalink / raw) To: openbmc On 6/29/20 9:03 PM, Bills, Jason M wrote: > > > On 6/29/2020 1:07 AM, Ratan Gupta wrote: >> >> On 6/26/20 12:25 AM, Bills, Jason M wrote: >>> >>> >>> On 6/25/2020 10:26 AM, Brad Bishop wrote: >>>> One idea floating around to address these is inventing a journal >>>> metadata scheme that is management interface agnostic. I >>>> understand the >>>> motivation behind that - it is much simpler for an application to >>>> slide >>>> a single journal entry into the journal with all the metadata >>>> needed to >>>> generate events, than it is for an application to snoop multiple >>>> signals >>>> off dbus and pull events out of that. >>>> >>>> But I wonder if inventing a management interface agnostic scheme for >>>> adding events to the journal is really just inventing a new data model >>>> for how we represent things in a server - e.g. are we just working >>>> around our dbus data model? Maybe we should fix it instead, so >>>> that it >>>> isn't so difficult for applications to use it? With that said I don't >>>> know how to do this and I could use more concrete details on which >>>> areas >>>> of the data model make it hard to consume signals. Does anyone >>>> have any >>>> ideas or examples? >>>> >>> >>> On this, I think we may want to separate logging vs. eventing both >>> in this feature discussion and in the tools we want to use. >>> >>> When we were talking about logging, I think the journal made sense >>> since it is designed for logging and has benefits around that usage. >>> However, I agree that it doesn't seem like the right tool for >>> sending and receiving events and signals and that D-Bus sounds like >>> the right tool for that. >>> >>> I think I'm still a little confused at the scope. My understanding >>> was that this initial design for EventService was only for >>> monitoring event messages and not resources in general. It seems >>> like it may not make sense to try to use the same tools and approach >>> for both message monitoring and resource monitoring? Do we need to >>> treat them separately for now to simplify the discussion? >> Jason, When you say event messages? What do you mean, Do you mean to >> say "/redfish/v1/Systems/system/Logservices/eventlog"? > >> If yes then this should also go as Resource Event, When ever any log >> entry gets created under System Log >> (/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would >> notify to the Redfish client saying that "ResourceCreated" with the >> URL of the Resource. > Yes, new entries under > "/redfish/v1/Systems/system/Logservices/eventlog", but I thought you > could register for specific MessageIDs, so it's not just a generic > "new resource" event like others would be. Can we register for MessageID? I thought client can register for whole registry not a specific Message ID. > > >> >> After receiving this event Redfish client will do a GET request on >> the URL(retrieved as part of event) to get the content of the log. >> >> This will become generic infra for all types of events. > What I'm saying is I don't know if there is a good generic solution to > cover both the EventLog and all other resources. I believe the > current EventService implementation was designed only for EventLog and > may not work well for generic resource events. Can you get me the example payload for EventLog which is going to be sent with the current design? I am not sure how the eventlog and other resources are different. For eventLogs also we have the associated D-bus objects(/xyz/openbmc_project/logging,/xyz/openbmc_project/dump etc) > >> >> I would be coming up with few design approaches and downside with >> each approach to take it to conclusion. > Thanks! What I'm proposing is that we clarify or possibly separate > the discussions about EventLog vs. generic resources to avoid > confusion and come up with the right solutions for each. > >> >> Ratan >> ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-07-03 10:15 ` Ratan Gupta @ 2020-07-06 21:30 ` Bills, Jason M 2020-07-13 6:32 ` Ratan Gupta 0 siblings, 1 reply; 44+ messages in thread From: Bills, Jason M @ 2020-07-06 21:30 UTC (permalink / raw) To: openbmc On 7/3/2020 3:15 AM, Ratan Gupta wrote: >>>> I think I'm still a little confused at the scope. My understanding >>>> was that this initial design for EventService was only for >>>> monitoring event messages and not resources in general. It seems >>>> like it may not make sense to try to use the same tools and approach >>>> for both message monitoring and resource monitoring? Do we need to >>>> treat them separately for now to simplify the discussion? >>> Jason, When you say event messages? What do you mean, Do you mean to >>> say "/redfish/v1/Systems/system/Logservices/eventlog"? > >>> If yes then this should also go as Resource Event, When ever any log >>> entry gets created under System Log >>> (/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would >>> notify to the Redfish client saying that "ResourceCreated" with the >>> URL of the Resource. >> Yes, new entries under >> "/redfish/v1/Systems/system/Logservices/eventlog", but I thought you >> could register for specific MessageIDs, so it's not just a generic >> "new resource" event like others would be. > > Can we register for MessageID? I thought client can register for whole > registry not a specific Message ID. > I don't really know. I thought that's what the current implementation allowed, but I don't know for sure if it can or should. >> >> >>> >>> After receiving this event Redfish client will do a GET request on >>> the URL(retrieved as part of event) to get the content of the log. >>> >>> This will become generic infra for all types of events. >> What I'm saying is I don't know if there is a good generic solution to >> cover both the EventLog and all other resources. I believe the >> current EventService implementation was designed only for EventLog and >> may not work well for generic resource events. > > Can you get me the example payload for EventLog which is going to be > sent with the current design? I am not sure how the eventlog and other > resources are different. > This is based on the assumption that for a LogService, you can register for a MessageId. If this is not possible, then they might be treated the same. > For eventLogs also we have the associated D-bus > objects(/xyz/openbmc_project/logging,/xyz/openbmc_project/dump etc) > For Intel platforms, we don't use /xyz/openbmc_project/logging, so we don't have D-Bus objects associated with each EventLog LogEntry. We use rsyslog to create a file that contains many LogEntries. However, as an unrelated side-thought: linking logging to /xyz/openbmc_project/dump made me wonder if there is a possible solution to the logging issue if we treat /xyz/openbmc_project/logging like /xyz/openbmc_project/dump and place a pointer to the log in the D-Bus object instead of the log itself? >> >>> >>> I would be coming up with few design approaches and downside with >>> each approach to take it to conclusion. >> Thanks! What I'm proposing is that we clarify or possibly separate >> the discussions about EventLog vs. generic resources to avoid >> confusion and come up with the right solutions for each. >> >>> >>> Ratan >>> ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-07-06 21:30 ` Bills, Jason M @ 2020-07-13 6:32 ` Ratan Gupta 2020-07-14 21:08 ` James Feist 0 siblings, 1 reply; 44+ messages in thread From: Ratan Gupta @ 2020-07-13 6:32 UTC (permalink / raw) To: openbmc, James Feist, Jason >> Bills, Jason M Hi James, As you asked for the examples in the thread(https://lists.ozlabs.org/pipermail/openbmc/2020-June/022125.html), I have created the mapping at the following location. https://gist.github.com/ratagupt/0aa4da098a60d49af90a7e4a6ea6d5f2 1) Map1: Mapping between redfish resources to Dbus resources 2) Map2: Mapping between redfish resource types to the ineterested Dbus interfaces 3) Map3: Mapping between Dbus resources to redfish resources I tried to cover the following scenario in the above mapping. * Redfish resource is mapped to multiple Dbus Resources * Redfish Property is mapped to single Dbus property * Redfish Property(complex property) is mapped to multiple dbus property. * Same type of Redfish Resources are mapped to different Dbus Resources * Redfish node url having multiple regex : Yet to take a look on this. Flow would be like as below => In bmcweb each Redfish node represents to a Redfish Resource. => Each node will be having it's own mapping between Redfish properties and the Dbus Resources. => Some code on bmcweb will walkthrough on each node during startup , get this mapping from each node and generate two mappings 1) Reverse mapping (Dbus Resource to Redfish Resource)(MAP3) and 2) mapping between Resource Types to the interested Dbus interfaces(MAP2) => To start with we will support few resource types and then scale it up as needed. => Map2 would be used when the Redfish client subscribe for the ResourceType to get the Dbus mappings. => Map3 would be used when the Dbus signal gets generated and need the Redfish mappings. => Once we have all thsese mapping gets generated and loaded into the memory, bmcweb would start listening for the interfaces listed in map2. => Once any Dbus signal gets generated map3 would be used to get the Redfish mapping. Please let me know if you have any concerns with this approach. Ratan On 7/7/20 3:00 AM, Bills, Jason M wrote: > > > On 7/3/2020 3:15 AM, Ratan Gupta wrote: >>>>> I think I'm still a little confused at the scope. My >>>>> understanding was that this initial design for EventService was >>>>> only for monitoring event messages and not resources in general. >>>>> It seems like it may not make sense to try to use the same tools >>>>> and approach for both message monitoring and resource monitoring? >>>>> Do we need to treat them separately for now to simplify the >>>>> discussion? >>>> Jason, When you say event messages? What do you mean, Do you mean >>>> to say "/redfish/v1/Systems/system/Logservices/eventlog"? > >>>> If yes then this should also go as Resource Event, When ever any >>>> log entry gets created under System Log >>>> (/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC >>>> would notify to the Redfish client saying that "ResourceCreated" >>>> with the URL of the Resource. >>> Yes, new entries under >>> "/redfish/v1/Systems/system/Logservices/eventlog", but I thought you >>> could register for specific MessageIDs, so it's not just a generic >>> "new resource" event like others would be. >> >> Can we register for MessageID? I thought client can register for >> whole registry not a specific Message ID. >> > I don't really know. I thought that's what the current implementation > allowed, but I don't know for sure if it can or should. > >>> >>> >>>> >>>> After receiving this event Redfish client will do a GET request on >>>> the URL(retrieved as part of event) to get the content of the log. >>>> >>>> This will become generic infra for all types of events. >>> What I'm saying is I don't know if there is a good generic solution >>> to cover both the EventLog and all other resources. I believe the >>> current EventService implementation was designed only for EventLog >>> and may not work well for generic resource events. >> >> Can you get me the example payload for EventLog which is going to be >> sent with the current design? I am not sure how the eventlog and >> other resources are different. >> > This is based on the assumption that for a LogService, you can > register for a MessageId. If this is not possible, then they might be > treated the same. > >> For eventLogs also we have the associated D-bus >> objects(/xyz/openbmc_project/logging,/xyz/openbmc_project/dump etc) >> > For Intel platforms, we don't use /xyz/openbmc_project/logging, so we > don't have D-Bus objects associated with each EventLog LogEntry. We > use rsyslog to create a file that contains many LogEntries. > > However, as an unrelated side-thought: linking logging to > /xyz/openbmc_project/dump made me wonder if there is a possible > solution to the logging issue if we treat /xyz/openbmc_project/logging > like /xyz/openbmc_project/dump and place a pointer to the log in the > D-Bus object instead of the log itself? > >>> >>>> >>>> I would be coming up with few design approaches and downside with >>>> each approach to take it to conclusion. >>> Thanks! What I'm proposing is that we clarify or possibly separate >>> the discussions about EventLog vs. generic resources to avoid >>> confusion and come up with the right solutions for each. >>> >>>> >>>> Ratan >>>> ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-07-13 6:32 ` Ratan Gupta @ 2020-07-14 21:08 ` James Feist 2020-07-30 9:10 ` Ratan Gupta 0 siblings, 1 reply; 44+ messages in thread From: James Feist @ 2020-07-14 21:08 UTC (permalink / raw) To: Ratan Gupta, openbmc, Jason >> Bills, Jason M, Puli, Apparao On 7/12/2020 11:32 PM, Ratan Gupta wrote: > Hi James, > > As you asked for the examples in the > thread(https://lists.ozlabs.org/pipermail/openbmc/2020-June/022125.html), I > have created the mapping at the following location. > > https://gist.github.com/ratagupt/0aa4da098a60d49af90a7e4a6ea6d5f2 Thanks for the examples. > > 1) Map1: Mapping between redfish resources to Dbus resources How does this work when the mapping isn't 1:1 vs D-Bus? Most of the time the enums to not match the d-bus enum, or take multiple d-bus interface to distinguish what the value should be. Also how does this work for discovered things, like multiple chassis? > 2) Map2: Mapping between redfish resource types to the ineterested Dbus > interfaces > 3) Map3: Mapping between Dbus resources to redfish resources > > > I tried to cover the following scenario in the above mapping. > > * Redfish resource is mapped to multiple Dbus Resources > * Redfish Property is mapped to single Dbus property > * Redfish Property(complex property) is mapped to multiple dbus property. > * Same type of Redfish Resources are mapped to different Dbus Resources > * Redfish node url having multiple regex : Yet to take a look on this. > > > Flow would be like as below > > => In bmcweb each Redfish node represents to a Redfish Resource. > => Each node will be having it's own mapping between Redfish properties > and the Dbus Resources. > > => Some code on bmcweb will walkthrough on each node during startup , > get this mapping from each node and generate > two mappings > 1) Reverse mapping (Dbus Resource to Redfish Resource)(MAP3) and > 2) mapping between Resource Types to the interested Dbus > interfaces(MAP2) > > => To start with we will support few resource types and then scale it up > as needed. I think we need an idea of what the final solution will look like for more complicated properties, or we'll be creating something that isn't future proof. > > => Map2 would be used when the Redfish client subscribe for the > ResourceType to get the Dbus mappings. > > => Map3 would be used when the Dbus signal gets generated and need the > Redfish mappings. Why can't these be the same mapping? I think having 3 different maps makes this very confusing. I also think this is attempting to generalize the problem too early. If you look at the Redfish code to determine some of the more complicated properties, sometimes it takes quite a bit of logic. That logic also would possibly be nice to reuse. Maybe we can take your idea of a match with a callback to some of the already existing property parsing? Obviously it would need some cleaning up, but I could see something with a map of schemas/properties to function pointers for property parsing. > > => Once we have all thsese mapping gets generated and loaded into the > memory, bmcweb would start listening > for the interfaces listed in map2. Why aren't these compiled in? I don't see why they need to be loaded, could just be in code. Also, they shouldn't be added as matches until there is a subscriber, or we'll add many unneeded matches. > > => Once any Dbus signal gets generated map3 would be used to get the > Redfish mapping. > > Please let me know if you have any concerns with this approach. > > Ratan ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation 2020-07-14 21:08 ` James Feist @ 2020-07-30 9:10 ` Ratan Gupta 0 siblings, 0 replies; 44+ messages in thread From: Ratan Gupta @ 2020-07-30 9:10 UTC (permalink / raw) To: openbmc Hi James, On 7/15/20 2:38 AM, James Feist wrote: > On 7/12/2020 11:32 PM, Ratan Gupta wrote: >> Hi James, >> >> As you asked for the examples in the >> thread(https://lists.ozlabs.org/pipermail/openbmc/2020-June/022125.html), >> I have created the mapping at the following location. >> >> https://gist.github.com/ratagupt/0aa4da098a60d49af90a7e4a6ea6d5f2 > > Thanks for the examples. > >> >> 1) Map1: Mapping between redfish resources to Dbus resources > > How does this work when the mapping isn't 1:1 vs D-Bus? Most of the > time the enums to not match the d-bus enum, or take multiple d-bus > interface to distinguish what the value should be. Also how does this > work for discovered things, like multiple chassis? I have not mentioned in the below use cases, but just look at the url https://gist.github.com/ratagupt/0aa4da098a60d49af90a7e4a6ea6d5f2#file-gistfile1-txt-L77 <https://gist.github.com/ratagupt/0aa4da098a60d49af90a7e4a6ea6d5f2#file-gistfile1-txt-L77> There we are covering the resources which are getting added dynamically. > > >> 2) Map2: Mapping between redfish resource types to the ineterested >> Dbus interfaces >> 3) Map3: Mapping between Dbus resources to redfish resources >> >> >> I tried to cover the following scenario in the above mapping. >> >> * Redfish resource is mapped to multiple Dbus Resources >> * Redfish Property is mapped to single Dbus property >> * Redfish Property(complex property) is mapped to multiple dbus >> property. >> * Same type of Redfish Resources are mapped to different Dbus Resources >> * Redfish node url having multiple regex : Yet to take a look on this. >> > >> Flow would be like as below >> >> => In bmcweb each Redfish node represents to a Redfish Resource. >> => Each node will be having it's own mapping between Redfish >> properties and the Dbus Resources. >> >> => Some code on bmcweb will walkthrough on each node during startup , >> get this mapping from each node and generate >> two mappings >> 1) Reverse mapping (Dbus Resource to Redfish Resource)(MAP3) and >> 2) mapping between Resource Types to the interested Dbus >> interfaces(MAP2) >> >> => To start with we will support few resource types and then scale it >> up as needed. > > I think we need an idea of what the final solution will look like for > more complicated properties, or we'll be creating something that isn't > future proof. I thought I covered the complicated one in the examples(gist link above) but just let me know the other redfish resource which I can take it in the examples. > >> >> => Map2 would be used when the Redfish client subscribe for the >> ResourceType to get the Dbus mappings. >> >> => Map3 would be used when the Dbus signal gets generated and need >> the Redfish mappings. > > Why can't these be the same mapping? To start monitoring for Dbus : We need Dbus Resource Path, Once we get any Dbus signal, we need to send redfish event which needs redfish resource path hence we need reverse map Did I get you correctly? > I think having 3 different maps makes this very confusing. I also > think this is attempting to generalize the problem too early. If you > look at the Redfish code to determine some of the more complicated > properties, sometimes it takes quite a bit of logic. That logic also > would possibly be nice to reuse. Maybe we can take your idea of a > match with a callback to some of the already existing property > parsing? Obviously it would need some cleaning up, but I could see > something with a map of schemas/properties to function pointers for > property parsing. > >> >> => Once we have all thsese mapping gets generated and loaded into the >> memory, bmcweb would start listening >> for the interfaces listed in map2. > > Why aren't these compiled in? I don't see why they need to be loaded, > could just be in code. Also, they shouldn't be added as matches until > there is a subscriber, or we'll add many unneeded matches. They would be compiled in, they would be in memory(big std::map). Agree monitoring should start only when there is a subscriber. > >> >> => Once any Dbus signal gets generated map3 would be used to get the >> Redfish mapping. >> >> Please let me know if you have any concerns with this approach. >> >> Ratan ^ permalink raw reply [flat|nested] 44+ messages in thread
end of thread, other threads:[~2020-07-30 9:10 UTC | newest] Thread overview: 44+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-01-31 20:53 Redfish EventService Implementation RAJESWARAN THILLAIGOVINDAN 2020-02-09 20:22 ` RAJESWARAN THILLAIGOVINDAN 2020-02-17 20:11 ` RAJESWARAN THILLAIGOVINDAN 2020-02-19 19:19 ` Puli, Apparao 2020-02-24 6:37 ` Ratan Gupta 2020-02-25 14:06 ` Puli, Apparao 2020-05-05 11:43 ` RAJESWARAN THILLAIGOVINDAN 2020-05-26 12:20 ` RAJESWARAN THILLAIGOVINDAN 2020-05-27 3:48 ` Puli, Apparao 2020-05-27 11:50 ` RAJESWARAN THILLAIGOVINDAN 2020-05-27 18:58 ` Puli, Apparao 2020-05-28 13:26 ` Ratan Gupta 2020-05-29 15:45 ` Redfish event log for new local user addition Puli, Apparao 2020-06-02 6:30 ` Ratan Gupta 2020-06-08 21:08 ` Redfish EventService Implementation Brad Bishop 2020-06-09 0:58 ` James Feist 2020-06-15 12:50 ` Ratan Gupta 2020-06-15 21:42 ` James Feist 2020-06-16 15:24 ` Patrick Williams 2020-06-16 16:11 ` James Feist 2020-06-17 12:08 ` Ratan Gupta 2020-06-17 17:16 ` James Feist 2020-06-17 17:19 ` Bills, Jason M 2020-06-17 18:30 ` Andrew Geissler 2020-06-17 20:45 ` Patrick Williams 2020-06-19 13:26 ` Ratan Gupta 2020-06-19 22:19 ` Bills, Jason M 2020-06-23 7:27 ` Ratan Gupta 2020-06-24 16:24 ` James Feist 2020-06-24 20:39 ` Patrick Williams 2020-06-25 13:45 ` Brad Bishop 2020-06-25 16:42 ` James Feist 2020-06-25 15:49 ` Brad Bishop 2020-06-25 16:44 ` James Feist 2020-06-25 17:26 ` Brad Bishop 2020-06-25 18:55 ` Bills, Jason M 2020-06-29 8:00 ` Ratan Gupta 2020-06-29 8:07 ` Ratan Gupta 2020-06-29 15:33 ` Bills, Jason M 2020-07-03 10:15 ` Ratan Gupta 2020-07-06 21:30 ` Bills, Jason M 2020-07-13 6:32 ` Ratan Gupta 2020-07-14 21:08 ` James Feist 2020-07-30 9:10 ` Ratan Gupta
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.