* Redfish EventService Implementation
@ 2020-01-31 20:53 RAJESWARAN THILLAIGOVINDAN
2020-02-09 20:22 ` RAJESWARAN THILLAIGOVINDAN
2020-06-08 21:08 ` Redfish EventService Implementation Brad Bishop
0 siblings, 2 replies; 44+ messages in thread
From: RAJESWARAN THILLAIGOVINDAN @ 2020-01-31 20:53 UTC (permalink / raw)
To: openbmc, apparao.puli
Hi,
I am going through the bmcweb code for implementing Redfish EventService
based on the design document
https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design
is hooked to the journal based Redfish Event Logging. For life cycle
events(ResourceAdded, ResourceRemoved, ResourceUpdated), using D-Bus
match, bmcweb can create an event log. This requires a JSON dictionary,
comprising an array of Redfish Resource Name and the D-Bus path. This
approach works only in case of one to one mapping of Redfish Resource
Name and the D-Bus path. For propertiesChanged events, if the Redfish
Resource property is not on the same D-Bus path or the Redfish Resource
property name is different from the D-Bus property name, then an
additional JSON dictionary to maintain this information is required.
With D-Bus match alone in the bmcweb, Redfish EventService can't be
fully supported. For the Message Registers and the Resource Types that
are supported, the relevant OpenBMC application must create an event log
in the journal using either the phosphor::logging::entry or
sd_journal_send() command.
After realizing that with D-Bus match in the bmcweb alone can't help to
fully implement EventService, I prefer to avoid using D-Bus match in
bmcweb. Instead, I prefer to modify the OpenBMC application that
generated the event to create an event log in the journal. Do you see
any advantage of using combination of D-Bus match in the bmcweb wherever
it is possible and changes to OpenBMC application in other cases to
create an event log ?
Your views are highly appreciated.
Thanks,
Rajes
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-01-31 20:53 Redfish EventService Implementation RAJESWARAN THILLAIGOVINDAN
@ 2020-02-09 20:22 ` RAJESWARAN THILLAIGOVINDAN
2020-02-17 20:11 ` RAJESWARAN THILLAIGOVINDAN
2020-02-19 19:19 ` Puli, Apparao
2020-06-08 21:08 ` Redfish EventService Implementation Brad Bishop
1 sibling, 2 replies; 44+ messages in thread
From: RAJESWARAN THILLAIGOVINDAN @ 2020-02-09 20:22 UTC (permalink / raw)
To: openbmc, apparao.puli, James Feist
ApparaRao.
As you have shown interest in this feature and submitted the design
document, do you have any opinion on this? Do you see any merit in using
D-Bus match in bmcweb to create event logs for life cycle events?
Please feel free to weigh in.
Thanks,
Rajes
On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
> Hi,
>
> I am going through the bmcweb code for implementing Redfish
> EventService based on the design document
> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design
> is hooked to the journal based Redfish Event Logging. For life cycle
> events(ResourceAdded, ResourceRemoved, ResourceUpdated), using D-Bus
> match, bmcweb can create an event log. This requires a JSON
> dictionary, comprising an array of Redfish Resource Name and the D-Bus
> path. This approach works only in case of one to one mapping of
> Redfish Resource Name and the D-Bus path. For propertiesChanged
> events, if the Redfish Resource property is not on the same D-Bus path
> or the Redfish Resource property name is different from the D-Bus
> property name, then an additional JSON dictionary to maintain this
> information is required. With D-Bus match alone in the bmcweb, Redfish
> EventService can't be fully supported. For the Message Registers and
> the Resource Types that are supported, the relevant OpenBMC
> application must create an event log in the journal using either the
> phosphor::logging::entry or sd_journal_send() command.
>
> After realizing that with D-Bus match in the bmcweb alone can't help
> to fully implement EventService, I prefer to avoid using D-Bus match
> in bmcweb. Instead, I prefer to modify the OpenBMC application that
> generated the event to create an event log in the journal. Do you see
> any advantage of using combination of D-Bus match in the bmcweb
> wherever it is possible and changes to OpenBMC application in other
> cases to create an event log ?
>
> Your views are highly appreciated.
>
> Thanks,
> Rajes
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-02-09 20:22 ` RAJESWARAN THILLAIGOVINDAN
@ 2020-02-17 20:11 ` RAJESWARAN THILLAIGOVINDAN
2020-02-19 19:19 ` Puli, Apparao
1 sibling, 0 replies; 44+ messages in thread
From: RAJESWARAN THILLAIGOVINDAN @ 2020-02-17 20:11 UTC (permalink / raw)
To: openbmc
I have implemented the skeleton code and uploaded it for review. Kindly
review and let me know your comments. For prototyping, based on the
Redfish Event Logging design, I have modified phosphosr-user-manager
application to log resource creation event when an account is created.
For reading the redfish event logs from the journal and writing to BMC
filesystem(/var/log/redfish), I have pulled the rsyslog configuration
from the OEM (Intel) and made minor changes.
These patches are work in progress and needs lot more changes. Any
suggestions regarding the approach is appreciated.
https://gerrit.openbmc-project.xyz/c/openbmc/openbmc/+/29464
https://gerrit.openbmc-project.xyz/c/openbmc/phosphor-user-manager/+/29463
https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/29462
Thanks,
Rajes
On 10-02-2020 01:52, RAJESWARAN THILLAIGOVINDAN wrote:
> ApparaRao.
>
> As you have shown interest in this feature and submitted the design
> document, do you have any opinion on this? Do you see any merit in
> using D-Bus match in bmcweb to create event logs for life cycle
> events? Please feel free to weigh in.
>
> Thanks,
> Rajes
>
> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>> Hi,
>>
>> I am going through the bmcweb code for implementing Redfish
>> EventService based on the design document
>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This
>> design is hooked to the journal based Redfish Event Logging. For life
>> cycle events(ResourceAdded, ResourceRemoved, ResourceUpdated), using
>> D-Bus match, bmcweb can create an event log. This requires a JSON
>> dictionary, comprising an array of Redfish Resource Name and the
>> D-Bus path. This approach works only in case of one to one mapping of
>> Redfish Resource Name and the D-Bus path. For propertiesChanged
>> events, if the Redfish Resource property is not on the same D-Bus
>> path or the Redfish Resource property name is different from the
>> D-Bus property name, then an additional JSON dictionary to maintain
>> this information is required. With D-Bus match alone in the bmcweb,
>> Redfish EventService can't be fully supported. For the Message
>> Registers and the Resource Types that are supported, the relevant
>> OpenBMC application must create an event log in the journal using
>> either the phosphor::logging::entry or sd_journal_send() command.
>>
>> After realizing that with D-Bus match in the bmcweb alone can't help
>> to fully implement EventService, I prefer to avoid using D-Bus match
>> in bmcweb. Instead, I prefer to modify the OpenBMC application that
>> generated the event to create an event log in the journal. Do you see
>> any advantage of using combination of D-Bus match in the bmcweb
>> wherever it is possible and changes to OpenBMC application in other
>> cases to create an event log ?
>>
>> Your views are highly appreciated.
>>
>> Thanks,
>> Rajes
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-02-09 20:22 ` RAJESWARAN THILLAIGOVINDAN
2020-02-17 20:11 ` RAJESWARAN THILLAIGOVINDAN
@ 2020-02-19 19:19 ` Puli, Apparao
2020-02-24 6:37 ` Ratan Gupta
1 sibling, 1 reply; 44+ messages in thread
From: Puli, Apparao @ 2020-02-19 19:19 UTC (permalink / raw)
To: RAJESWARAN THILLAIGOVINDAN, openbmc, James Feist
Hi,
I am sorry for late response as this mail is buried under and got
struck back of my mind.
As i did mentioned in EventService design document, EventLog Monitor
service is not common across the organizations( Example: Intel uses the
redfish event logs file and inotify mechanism for monitoring the event
logs. Where as IBM uses d-bus event log mechanism and they can relay on
match rules). That said challenges with ResourceType mapping will be
different in both monitoring mechanisms. This is good point. Initially
when i started EventService design, i thought we can have mapping in
bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
per Intel design) but not sure that may become difficult when we expand
supported ResourceTypes.
As per my reading from below query, You are looking at d-bus match rules
and ResourceTypes mapping which is more specific to d-bus event
logging(IBM way of implementing event logging). reading it from journal
logs will give more information but that will impact the performance to
large extent. This might be one of the reason why we (Intel) uses
Redfish message ID while logging redfish events logs to journal(You can
refer design document for same at
https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md).
In opinion, in your d-bus if you are using some kind of filter(Example
REDFISH_MESSAGE_ID) while logging in journal logs for all events and
figure out the way to monitor the journal logs without impacting the
performance, that should be ok as long as match filters are satisfied
for Redfish EventService subscriptions and supported Types(Again differs
with implementation).
Thanks,
-Appu
On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
> ApparaRao.
>
> As you have shown interest in this feature and submitted the design
> document, do you have any opinion on this? Do you see any merit in
> using D-Bus match in bmcweb to create event logs for life cycle
> events? Please feel free to weigh in.
>
> Thanks,
> Rajes
>
> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>> Hi,
>>
>> I am going through the bmcweb code for implementing Redfish
>> EventService based on the design document
>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This
>> design is hooked to the journal based Redfish Event Logging. For life
>> cycle events(ResourceAdded, ResourceRemoved, ResourceUpdated), using
>> D-Bus match, bmcweb can create an event log. This requires a JSON
>> dictionary, comprising an array of Redfish Resource Name and the
>> D-Bus path. This approach works only in case of one to one mapping of
>> Redfish Resource Name and the D-Bus path. For propertiesChanged
>> events, if the Redfish Resource property is not on the same D-Bus
>> path or the Redfish Resource property name is different from the
>> D-Bus property name, then an additional JSON dictionary to maintain
>> this information is required. With D-Bus match alone in the bmcweb,
>> Redfish EventService can't be fully supported. For the Message
>> Registers and the Resource Types that are supported, the relevant
>> OpenBMC application must create an event log in the journal using
>> either the phosphor::logging::entry or sd_journal_send() command.
>>
>> After realizing that with D-Bus match in the bmcweb alone can't help
>> to fully implement EventService, I prefer to avoid using D-Bus match
>> in bmcweb. Instead, I prefer to modify the OpenBMC application that
>> generated the event to create an event log in the journal. Do you see
>> any advantage of using combination of D-Bus match in the bmcweb
>> wherever it is possible and changes to OpenBMC application in other
>> cases to create an event log ?
>>
>> Your views are highly appreciated.
>>
>> Thanks,
>> Rajes
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-02-19 19:19 ` Puli, Apparao
@ 2020-02-24 6:37 ` Ratan Gupta
2020-02-25 14:06 ` Puli, Apparao
0 siblings, 1 reply; 44+ messages in thread
From: Ratan Gupta @ 2020-02-24 6:37 UTC (permalink / raw)
To: openbmc, Puli, Apparao
[-- Attachment #1: Type: text/plain, Size: 5454 bytes --]
Hi Apparao,
On 2/20/20 12:49 AM, Puli, Apparao wrote:
> Hi,
>
> I am sorry for late response as this mail is buried under and got
> struck back of my mind.
>
> As i did mentioned in EventService design document, EventLog Monitor
> service is not common across the organizations( Example: Intel uses
> the redfish event logs file and inotify mechanism for monitoring the
> event logs. Where as IBM uses d-bus event log mechanism and they can
> relay on match rules). That said challenges with ResourceType mapping
> will be different in both monitoring mechanisms. This is good point.
> Initially when i started EventService design, i thought we can have
> mapping in bmcweb for ResourceTypes with set of MessageID's for Logged
> events ( As per Intel design) but not sure that may become difficult
> when we expand supported ResourceTypes.
If I am getting correctly, Here is the flow which Intel uses.
1. Individual repo have to push the logs using sd_journal_send which
will write to the file(/var/log/redfish) by using rsyslog daemon
sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
"REDFISH_MESSAGE_ID=%s",
"ResourceEvent.1.0.ResourceCreated",NULL);
* How you would populate the "OriginOfCondition" during sending of
event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
* Any plan to add resource path in this journal message which
tells that this is the path for which resource created event
generated.
* Would the path be "Redfish Path/ D-bus Path"?
* Where the mapping would be done between D-busPath/Redfish
Resource path?
Cons: Every application have to make the change(use sd_journal_send).
My thought is backend application should not be aware of the redfish terminlogy.
*2.* Some application(bmcweb) would do the Inotify on the
path(/var/log/redfish) and send the event once there is any activity on
this file.
> I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
per Intel design)
Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
>
> As per my reading from below query, You are looking at d-bus match
> rules and ResourceTypes mapping which is more specific to d-bus event
> logging(IBM way of implementing event logging). reading it from
> journal logs will give more information but that will impact the
> performance to large extent. This might be one of the reason why we
> (Intel) uses Redfish message ID while logging redfish events logs to
> journal(You can refer design document for same at
> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md).
> In opinion, in your d-bus if you are using some kind of filter(Example
> REDFISH_MESSAGE_ID) while logging in journal logs for all events and
> figure out the way to monitor the journal logs without impacting the
> performance, that should be ok as long as match filters are satisfied
> for Redfish EventService subscriptions and supported Types(Again
> differs with implementation).
>
> Thanks,
>
> -Appu
>
> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
>> ApparaRao.
>>
>> As you have shown interest in this feature and submitted the design
>> document, do you have any opinion on this? Do you see any merit in
>> using D-Bus match in bmcweb to create event logs for life cycle
>> events? Please feel free to weigh in.
>>
>> Thanks,
>> Rajes
>>
>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>>> Hi,
>>>
>>> I am going through the bmcweb code for implementing Redfish
>>> EventService based on the design document
>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This
>>> design is hooked to the journal based Redfish Event Logging. For
>>> life cycle events(ResourceAdded, ResourceRemoved, ResourceUpdated),
>>> using D-Bus match, bmcweb can create an event log. This requires a
>>> JSON dictionary, comprising an array of Redfish Resource Name and
>>> the D-Bus path. This approach works only in case of one to one
>>> mapping of Redfish Resource Name and the D-Bus path. For
>>> propertiesChanged events, if the Redfish Resource property is not on
>>> the same D-Bus path or the Redfish Resource property name is
>>> different from the D-Bus property name, then an additional JSON
>>> dictionary to maintain this information is required. With D-Bus
>>> match alone in the bmcweb, Redfish EventService can't be fully
>>> supported. For the Message Registers and the Resource Types that are
>>> supported, the relevant OpenBMC application must create an event log
>>> in the journal using either the phosphor::logging::entry or
>>> sd_journal_send() command.
>>>
>>> After realizing that with D-Bus match in the bmcweb alone can't help
>>> to fully implement EventService, I prefer to avoid using D-Bus match
>>> in bmcweb. Instead, I prefer to modify the OpenBMC application that
>>> generated the event to create an event log in the journal. Do you
>>> see any advantage of using combination of D-Bus match in the bmcweb
>>> wherever it is possible and changes to OpenBMC application in other
>>> cases to create an event log ?
>>>
>>> Your views are highly appreciated.
>>>
>>> Thanks,
>>> Rajes
>>
Thanks
Ratan
[-- Attachment #2: Type: text/html, Size: 8088 bytes --]
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-02-24 6:37 ` Ratan Gupta
@ 2020-02-25 14:06 ` Puli, Apparao
2020-05-05 11:43 ` RAJESWARAN THILLAIGOVINDAN
2020-05-26 12:20 ` RAJESWARAN THILLAIGOVINDAN
0 siblings, 2 replies; 44+ messages in thread
From: Puli, Apparao @ 2020-02-25 14:06 UTC (permalink / raw)
To: Ratan Gupta, openbmc
[-- Attachment #1: Type: text/plain, Size: 7102 bytes --]
Hi Ratan,
Comments in-line
On 2/24/2020 12:07 PM, Ratan Gupta wrote:
>
> Hi Apparao,
>
> On 2/20/20 12:49 AM, Puli, Apparao wrote:
>> Hi,
>>
>> I am sorry for late response as this mail is buried under and got
>> struck back of my mind.
>>
>> As i did mentioned in EventService design document, EventLog Monitor
>> service is not common across the organizations( Example: Intel uses
>> the redfish event logs file and inotify mechanism for monitoring the
>> event logs. Where as IBM uses d-bus event log mechanism and they can
>> relay on match rules). That said challenges with ResourceType mapping
>> will be different in both monitoring mechanisms. This is good point.
>> Initially when i started EventService design, i thought we can have
>> mapping in bmcweb for ResourceTypes with set of MessageID's for
>> Logged events ( As per Intel design) but not sure that may become
>> difficult when we expand supported ResourceTypes.
>
> If I am getting correctly, Here is the flow which Intel uses.
>
> 1. Individual repo have to push the logs using sd_journal_send which
> will write to the file(/var/log/redfish) by using rsyslog daemon
>
> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
> "REDFISH_MESSAGE_ID=%s",
> "ResourceEvent.1.0.ResourceCreated",NULL);
>
> * How you would populate the "OriginOfCondition" during sending
> of event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
>
Currently in logServices( logEntry), we are not reporting the
"OriginOfCondition" property as per schema. I will check with Jason( Who
wrote the logService) and get back on this.
BTW can you give me how this information is fetched in IBM way of
LogService implementation( D-Bus)? If you already ratified any such, i
think we can leverage. If not, We work together for solution.
> * Any plan to add resource path in this journal message which
> tells that this is the path for which resource created event
> generated.
>
Same as above.
>
> * Would the path be "Redfish Path/ D-bus Path"?
>
As per Redfish specification, This should be "@odata.id" which means it
should be of resource uri and so we can't use d-bus path here for
OriginOfConditions.
>
> * Where the mapping would be done between D-busPath/Redfish
> Resource path?
>
>
> Cons: Every application have to make the change(use sd_journal_send).
> My thought is backend application should not be aware of the redfish terminlogy.
Having separate process only for this mapping may not be good( No
different from maintaining that map inside bmcweb as there won't be any
other consumers). Ideal way is, that should be mapped while logging
logEntry's itself. But we are not doing it currently which need to be
re-looked. Give me some time, I will think and check with other folks
and get back.
>
> *2.* Some application(bmcweb) would do the Inotify on the
> path(/var/log/redfish) and send the event once there is any activity
> on this file.
>
> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
> per Intel design)
>
> Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
Initially i thought "ResourceType" based event filtering can be done
using minimal mapping( Using MessageID and args). But that will work for
minimal set. If the supported ResourceTypes grows, we will have bigger
challenges which i can sense it now. Anyway, Supported Resources are
completely implementation specific. If this value is empty means, by
default all event logs will be sent to subscribers. This is what we can
start with before supported ResourceTypes list grows.
>>
>> As per my reading from below query, You are looking at d-bus match
>> rules and ResourceTypes mapping which is more specific to d-bus event
>> logging(IBM way of implementing event logging). reading it from
>> journal logs will give more information but that will impact the
>> performance to large extent. This might be one of the reason why we
>> (Intel) uses Redfish message ID while logging redfish events logs to
>> journal(You can refer design document for same at
>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md).
>> In opinion, in your d-bus if you are using some kind of
>> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs for
>> all events and figure out the way to monitor the journal logs without
>> impacting the performance, that should be ok as long as match filters
>> are satisfied for Redfish EventService subscriptions and supported
>> Types(Again differs with implementation).
>>
>> Thanks,
>>
>> -Appu
>>
>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
>>> ApparaRao.
>>>
>>> As you have shown interest in this feature and submitted the design
>>> document, do you have any opinion on this? Do you see any merit in
>>> using D-Bus match in bmcweb to create event logs for life cycle
>>> events? Please feel free to weigh in.
>>>
>>> Thanks,
>>> Rajes
>>>
>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>>>> Hi,
>>>>
>>>> I am going through the bmcweb code for implementing Redfish
>>>> EventService based on the design document
>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This
>>>> design is hooked to the journal based Redfish Event Logging. For
>>>> life cycle events(ResourceAdded, ResourceRemoved,
>>>> ResourceUpdated), using D-Bus match, bmcweb can create an event
>>>> log. This requires a JSON dictionary, comprising an array of
>>>> Redfish Resource Name and the D-Bus path. This approach works only
>>>> in case of one to one mapping of Redfish Resource Name and the
>>>> D-Bus path. For propertiesChanged events, if the Redfish Resource
>>>> property is not on the same D-Bus path or the Redfish Resource
>>>> property name is different from the D-Bus property name, then an
>>>> additional JSON dictionary to maintain this information is
>>>> required. With D-Bus match alone in the bmcweb, Redfish
>>>> EventService can't be fully supported. For the Message Registers
>>>> and the Resource Types that are supported, the relevant OpenBMC
>>>> application must create an event log in the journal using either
>>>> the phosphor::logging::entry or sd_journal_send() command.
>>>>
>>>> After realizing that with D-Bus match in the bmcweb alone can't
>>>> help to fully implement EventService, I prefer to avoid using D-Bus
>>>> match in bmcweb. Instead, I prefer to modify the OpenBMC
>>>> application that generated the event to create an event log in the
>>>> journal. Do you see any advantage of using combination of D-Bus
>>>> match in the bmcweb wherever it is possible and changes to OpenBMC
>>>> application in other cases to create an event log ?
>>>>
>>>> Your views are highly appreciated.
>>>>
>>>> Thanks,
>>>> Rajes
>>>
> Thanks
> Ratan
>
>
[-- Attachment #2: Type: text/html, Size: 11239 bytes --]
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-02-25 14:06 ` Puli, Apparao
@ 2020-05-05 11:43 ` RAJESWARAN THILLAIGOVINDAN
2020-05-26 12:20 ` RAJESWARAN THILLAIGOVINDAN
1 sibling, 0 replies; 44+ messages in thread
From: RAJESWARAN THILLAIGOVINDAN @ 2020-05-05 11:43 UTC (permalink / raw)
To: Puli, Apparao, Ratan Gupta, openbmc
[-- Attachment #1: Type: text/plain, Size: 8540 bytes --]
Further to my previous mail, I wanted to see how a JSON dictionary would
look like for implementing Redfish EventService based on D-Bus match.
The D-Bus match approach is good for implementing life cycle
events((ResourceAdded, ResourceRemoved, ResourceUpdated). In bmcweb, a
Redfish Resource is viewed as having a set of properties which comes
from one or more D-Bus objects. So, when a client subscribes for a
ResourceType, using a JSON dictionary, find the D-Bus object(s) mapped
to the ResourceType and create matches. When the D-Bus match occurs,
again using aJSON dictionary, find the Redfish ResourceType mapped to
the D-Bus object and create events. The JSON dictionary should also
provide the Redifsh URI which needs to be specified in the event. An
example JSON dictionary for mapping the Redifsh ResourceTypes LogEntry
and ComputerSystem is available here :
https://gist.github.com/trajeswaran/fec230abd36181f85d2f20d09164ec05. In
case of LogEntry, there is one to one mapping of ResourceType to D-Bus
object. In case of ComputerSystem, the properties comes from multiple
D-Bus objects. Do you see any drawback with this approach? Kindly let
me know what you think. Thanks in advance.
On 25-02-2020 19:36, Puli, Apparao wrote:
>
> Hi Ratan,
>
> Comments in-line
>
> On 2/24/2020 12:07 PM, Ratan Gupta wrote:
>>
>> Hi Apparao,
>>
>> On 2/20/20 12:49 AM, Puli, Apparao wrote:
>>> Hi,
>>>
>>> I am sorry for late response as this mail is buried under and got
>>> struck back of my mind.
>>>
>>> As i did mentioned in EventService design document, EventLog Monitor
>>> service is not common across the organizations( Example: Intel uses
>>> the redfish event logs file and inotify mechanism for monitoring the
>>> event logs. Where as IBM uses d-bus event log mechanism and they can
>>> relay on match rules). That said challenges with ResourceType
>>> mapping will be different in both monitoring mechanisms. This is
>>> good point. Initially when i started EventService design, i thought
>>> we can have mapping in bmcweb for ResourceTypes with set of
>>> MessageID's for Logged events ( As per Intel design) but not sure
>>> that may become difficult when we expand supported ResourceTypes.
>>
>> If I am getting correctly, Here is the flow which Intel uses.
>>
>> 1. Individual repo have to push the logs using sd_journal_send which
>> will write to the file(/var/log/redfish) by using rsyslog daemon
>>
>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
>> "REDFISH_MESSAGE_ID=%s",
>> "ResourceEvent.1.0.ResourceCreated",NULL);
>>
>> * How you would populate the "OriginOfCondition" during sending
>> of event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
>>
> Currently in logServices( logEntry), we are not reporting the
> "OriginOfCondition" property as per schema. I will check with Jason(
> Who wrote the logService) and get back on this.
>
> BTW can you give me how this information is fetched in IBM way of
> LogService implementation( D-Bus)? If you already ratified any such, i
> think we can leverage. If not, We work together for solution.
>
>> * Any plan to add resource path in this journal message which
>> tells that this is the path for which resource created event
>> generated.
>>
> Same as above.
>>
>> * Would the path be "Redfish Path/ D-bus Path"?
>>
> As per Redfish specification, This should be "@odata.id" which means
> it should be of resource uri and so we can't use d-bus path here for
> OriginOfConditions.
>>
>> * Where the mapping would be done between D-busPath/Redfish
>> Resource path?
>>
>>
>> Cons: Every application have to make the change(use sd_journal_send).
>> My thought is backend application should not be aware of the redfish terminlogy.
>
> Having separate process only for this mapping may not be good( No
> different from maintaining that map inside bmcweb as there won't be
> any other consumers). Ideal way is, that should be mapped while
> logging logEntry's itself. But we are not doing it currently which
> need to be re-looked. Give me some time, I will think and check with
> other folks and get back.
>
>> *2.* Some application(bmcweb) would do the Inotify on the
>> path(/var/log/redfish) and send the event once there is any activity
>> on this file.
>>
>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
>> per Intel design)
>>
>> Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
> Initially i thought "ResourceType" based event filtering can be done
> using minimal mapping( Using MessageID and args). But that will work
> for minimal set. If the supported ResourceTypes grows, we will have
> bigger challenges which i can sense it now. Anyway, Supported
> Resources are completely implementation specific. If this value is
> empty means, by default all event logs will be sent to subscribers.
> This is what we can start with before supported ResourceTypes list grows.
>>>
>>> As per my reading from below query, You are looking at d-bus match
>>> rules and ResourceTypes mapping which is more specific to d-bus
>>> event logging(IBM way of implementing event logging). reading it
>>> from journal logs will give more information but that will impact
>>> the performance to large extent. This might be one of the reason why
>>> we (Intel) uses Redfish message ID while logging redfish events logs
>>> to journal(You can refer design document for same at
>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md).
>>> In opinion, in your d-bus if you are using some kind of
>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs for
>>> all events and figure out the way to monitor the journal logs
>>> without impacting the performance, that should be ok as long as
>>> match filters are satisfied for Redfish EventService subscriptions
>>> and supported Types(Again differs with implementation).
>>>
>>> Thanks,
>>>
>>> -Appu
>>>
>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>> ApparaRao.
>>>>
>>>> As you have shown interest in this feature and submitted the design
>>>> document, do you have any opinion on this? Do you see any merit in
>>>> using D-Bus match in bmcweb to create event logs for life cycle
>>>> events? Please feel free to weigh in.
>>>>
>>>> Thanks,
>>>> Rajes
>>>>
>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>> Hi,
>>>>>
>>>>> I am going through the bmcweb code for implementing Redfish
>>>>> EventService based on the design document
>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This
>>>>> design is hooked to the journal based Redfish Event Logging. For
>>>>> life cycle events(ResourceAdded, ResourceRemoved,
>>>>> ResourceUpdated), using D-Bus match, bmcweb can create an event
>>>>> log. This requires a JSON dictionary, comprising an array of
>>>>> Redfish Resource Name and the D-Bus path. This approach works only
>>>>> in case of one to one mapping of Redfish Resource Name and the
>>>>> D-Bus path. For propertiesChanged events, if the Redfish Resource
>>>>> property is not on the same D-Bus path or the Redfish Resource
>>>>> property name is different from the D-Bus property name, then an
>>>>> additional JSON dictionary to maintain this information is
>>>>> required. With D-Bus match alone in the bmcweb, Redfish
>>>>> EventService can't be fully supported. For the Message Registers
>>>>> and the Resource Types that are supported, the relevant OpenBMC
>>>>> application must create an event log in the journal using either
>>>>> the phosphor::logging::entry or sd_journal_send() command.
>>>>>
>>>>> After realizing that with D-Bus match in the bmcweb alone can't
>>>>> help to fully implement EventService, I prefer to avoid using
>>>>> D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC
>>>>> application that generated the event to create an event log in the
>>>>> journal. Do you see any advantage of using combination of D-Bus
>>>>> match in the bmcweb wherever it is possible and changes to OpenBMC
>>>>> application in other cases to create an event log ?
>>>>>
>>>>> Your views are highly appreciated.
>>>>>
>>>>> Thanks,
>>>>> Rajes
>>>>
>> Thanks
>> Ratan
>>
>>
[-- Attachment #2: Type: text/html, Size: 13434 bytes --]
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-02-25 14:06 ` Puli, Apparao
2020-05-05 11:43 ` RAJESWARAN THILLAIGOVINDAN
@ 2020-05-26 12:20 ` RAJESWARAN THILLAIGOVINDAN
2020-05-27 3:48 ` Puli, Apparao
1 sibling, 1 reply; 44+ messages in thread
From: RAJESWARAN THILLAIGOVINDAN @ 2020-05-26 12:20 UTC (permalink / raw)
To: Puli, Apparao, Ratan Gupta, openbmc
[-- Attachment #1: Type: text/plain, Size: 8212 bytes --]
Apparao,
I see that you have uploaded a
patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) for
supporting server sent events. This patch supports event filtering based
on registry prefix and/or messageId.
I would like to know if you have any plan to support event filtering
based on resource type. If so, I would like to work together for a
better solution. Earlier, I have proposed a solution based on D-Bus
match using a dictionary. With that approach, the major challenge is to
map Redfish resource and properties to D-Bus object and properties
respectively. If D-Bus applications are modified to include resource
type and origin of condition in the event, then there is no need for a
map. But,that brings Redfish terminology to the application. Also, this
will become an overhead if an OEM is not interested in Redfish event
service.
Thanks,
Rajes
On 25-02-2020 19:36, Puli, Apparao wrote:
>
> Hi Ratan,
>
> Comments in-line
>
> On 2/24/2020 12:07 PM, Ratan Gupta wrote:
>>
>> Hi Apparao,
>>
>> On 2/20/20 12:49 AM, Puli, Apparao wrote:
>>> Hi,
>>>
>>> I am sorry for late response as this mail is buried under and got
>>> struck back of my mind.
>>>
>>> As i did mentioned in EventService design document, EventLog Monitor
>>> service is not common across the organizations( Example: Intel uses
>>> the redfish event logs file and inotify mechanism for monitoring the
>>> event logs. Where as IBM uses d-bus event log mechanism and they can
>>> relay on match rules). That said challenges with ResourceType
>>> mapping will be different in both monitoring mechanisms. This is
>>> good point. Initially when i started EventService design, i thought
>>> we can have mapping in bmcweb for ResourceTypes with set of
>>> MessageID's for Logged events ( As per Intel design) but not sure
>>> that may become difficult when we expand supported ResourceTypes.
>>
>> If I am getting correctly, Here is the flow which Intel uses.
>>
>> 1. Individual repo have to push the logs using sd_journal_send which
>> will write to the file(/var/log/redfish) by using rsyslog daemon
>>
>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
>> "REDFISH_MESSAGE_ID=%s",
>> "ResourceEvent.1.0.ResourceCreated",NULL);
>>
>> * How you would populate the "OriginOfCondition" during sending
>> of event? (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
>>
> Currently in logServices( logEntry), we are not reporting the
> "OriginOfCondition" property as per schema. I will check with Jason(
> Who wrote the logService) and get back on this.
>
> BTW can you give me how this information is fetched in IBM way of
> LogService implementation( D-Bus)? If you already ratified any such, i
> think we can leverage. If not, We work together for solution.
>
>> * Any plan to add resource path in this journal message which
>> tells that this is the path for which resource created event
>> generated.
>>
> Same as above.
>>
>> * Would the path be "Redfish Path/ D-bus Path"?
>>
> As per Redfish specification, This should be "@odata.id" which means
> it should be of resource uri and so we can't use d-bus path here for
> OriginOfConditions.
>>
>> * Where the mapping would be done between D-busPath/Redfish
>> Resource path?
>>
>>
>> Cons: Every application have to make the change(use sd_journal_send).
>> My thought is backend application should not be aware of the redfish terminlogy.
>
> Having separate process only for this mapping may not be good( No
> different from maintaining that map inside bmcweb as there won't be
> any other consumers). Ideal way is, that should be mapped while
> logging logEntry's itself. But we are not doing it currently which
> need to be re-looked. Give me some time, I will think and check with
> other folks and get back.
>
>> *2.* Some application(bmcweb) would do the Inotify on the
>> path(/var/log/redfish) and send the event once there is any activity
>> on this file.
>>
>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
>> per Intel design)
>>
>> Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
> Initially i thought "ResourceType" based event filtering can be done
> using minimal mapping( Using MessageID and args). But that will work
> for minimal set. If the supported ResourceTypes grows, we will have
> bigger challenges which i can sense it now. Anyway, Supported
> Resources are completely implementation specific. If this value is
> empty means, by default all event logs will be sent to subscribers.
> This is what we can start with before supported ResourceTypes list grows.
>>>
>>> As per my reading from below query, You are looking at d-bus match
>>> rules and ResourceTypes mapping which is more specific to d-bus
>>> event logging(IBM way of implementing event logging). reading it
>>> from journal logs will give more information but that will impact
>>> the performance to large extent. This might be one of the reason why
>>> we (Intel) uses Redfish message ID while logging redfish events logs
>>> to journal(You can refer design document for same at
>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md).
>>> In opinion, in your d-bus if you are using some kind of
>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs for
>>> all events and figure out the way to monitor the journal logs
>>> without impacting the performance, that should be ok as long as
>>> match filters are satisfied for Redfish EventService subscriptions
>>> and supported Types(Again differs with implementation).
>>>
>>> Thanks,
>>>
>>> -Appu
>>>
>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>> ApparaRao.
>>>>
>>>> As you have shown interest in this feature and submitted the design
>>>> document, do you have any opinion on this? Do you see any merit in
>>>> using D-Bus match in bmcweb to create event logs for life cycle
>>>> events? Please feel free to weigh in.
>>>>
>>>> Thanks,
>>>> Rajes
>>>>
>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>> Hi,
>>>>>
>>>>> I am going through the bmcweb code for implementing Redfish
>>>>> EventService based on the design document
>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This
>>>>> design is hooked to the journal based Redfish Event Logging. For
>>>>> life cycle events(ResourceAdded, ResourceRemoved,
>>>>> ResourceUpdated), using D-Bus match, bmcweb can create an event
>>>>> log. This requires a JSON dictionary, comprising an array of
>>>>> Redfish Resource Name and the D-Bus path. This approach works only
>>>>> in case of one to one mapping of Redfish Resource Name and the
>>>>> D-Bus path. For propertiesChanged events, if the Redfish Resource
>>>>> property is not on the same D-Bus path or the Redfish Resource
>>>>> property name is different from the D-Bus property name, then an
>>>>> additional JSON dictionary to maintain this information is
>>>>> required. With D-Bus match alone in the bmcweb, Redfish
>>>>> EventService can't be fully supported. For the Message Registers
>>>>> and the Resource Types that are supported, the relevant OpenBMC
>>>>> application must create an event log in the journal using either
>>>>> the phosphor::logging::entry or sd_journal_send() command.
>>>>>
>>>>> After realizing that with D-Bus match in the bmcweb alone can't
>>>>> help to fully implement EventService, I prefer to avoid using
>>>>> D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC
>>>>> application that generated the event to create an event log in the
>>>>> journal. Do you see any advantage of using combination of D-Bus
>>>>> match in the bmcweb wherever it is possible and changes to OpenBMC
>>>>> application in other cases to create an event log ?
>>>>>
>>>>> Your views are highly appreciated.
>>>>>
>>>>> Thanks,
>>>>> Rajes
>>>>
>> Thanks
>> Ratan
>>
>>
[-- Attachment #2: Type: text/html, Size: 13084 bytes --]
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-05-26 12:20 ` RAJESWARAN THILLAIGOVINDAN
@ 2020-05-27 3:48 ` Puli, Apparao
2020-05-27 11:50 ` RAJESWARAN THILLAIGOVINDAN
0 siblings, 1 reply; 44+ messages in thread
From: Puli, Apparao @ 2020-05-27 3:48 UTC (permalink / raw)
To: RAJESWARAN THILLAIGOVINDAN, Ratan Gupta, openbmc
[-- Attachment #1: Type: text/plain, Size: 9633 bytes --]
Hi Rajeswaran,
Thanks for your mail. At the moment, I don't have plans to support
the "ResourceTypes", "OriginResources" based filtering. Basically Intel
uses file systems based redfish event logs( journalctl -> rsync->
filesystem) and doesn't use D-Bus mechanism like IBM uses. So I am not
much familiar with D-Bus logging but some of the suggestions:
1) While logging redfish events over D-Bus itself, it can provide
details on ResourceTypes and OriginResource URI/Path.
This is ideal and most efficient way. Since we walked a walked
long distance from start, Its hard to modify all the services to uses
these 2 new input parameters while logging events( Requires change in
almost all repo's)
2) For resourcesTypes: Can have mapping dictionary against all
MessageId's. For OriginResources: I believe, event log over D-Bus is
already holding the Path. If so, last 3/4 nodes of uri can be taken and
mapped against the resources and that can be used in Event filtering. We
did used same mechanism in case of telemetry while mapping
MetricReportDefinitions to URI.
Hope this helps.
Thanks,
-Appu
On 5/26/2020 5:50 PM, RAJESWARAN THILLAIGOVINDAN wrote:
>
> Apparao,
>
> I see that you have uploaded a
> patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441) for
> supporting server sent events. This patch supports event filtering
> based on registry prefix and/or messageId.
>
> I would like to know if you have any plan to support event filtering
> based on resource type. If so, I would like to work together for a
> better solution. Earlier, I have proposed a solution based on D-Bus
> match using a dictionary. With that approach, the major challenge is
> to map Redfish resource and properties to D-Bus object and properties
> respectively. If D-Bus applications are modified to include resource
> type and origin of condition in the event, then there is no need for a
> map. But,that brings Redfish terminology to the application. Also,
> this will become an overhead if an OEM is not interested in Redfish
> event service.
>
> Thanks,
> Rajes
> On 25-02-2020 19:36, Puli, Apparao wrote:
>>
>> Hi Ratan,
>>
>> Comments in-line
>>
>> On 2/24/2020 12:07 PM, Ratan Gupta wrote:
>>>
>>> Hi Apparao,
>>>
>>> On 2/20/20 12:49 AM, Puli, Apparao wrote:
>>>> Hi,
>>>>
>>>> I am sorry for late response as this mail is buried under and got
>>>> struck back of my mind.
>>>>
>>>> As i did mentioned in EventService design document, EventLog
>>>> Monitor service is not common across the organizations( Example:
>>>> Intel uses the redfish event logs file and inotify mechanism for
>>>> monitoring the event logs. Where as IBM uses d-bus event log
>>>> mechanism and they can relay on match rules). That said challenges
>>>> with ResourceType mapping will be different in both monitoring
>>>> mechanisms. This is good point. Initially when i started
>>>> EventService design, i thought we can have mapping in bmcweb for
>>>> ResourceTypes with set of MessageID's for Logged events ( As per
>>>> Intel design) but not sure that may become difficult when we expand
>>>> supported ResourceTypes.
>>>
>>> If I am getting correctly, Here is the flow which Intel uses.
>>>
>>> 1. Individual repo have to push the logs using sd_journal_send
>>> which will write to the file(/var/log/redfish) by using rsyslog
>>> daemon
>>>
>>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
>>> "REDFISH_MESSAGE_ID=%s",
>>> "ResourceEvent.1.0.ResourceCreated",NULL);
>>>
>>> * How you would populate the "OriginOfCondition" during
>>> sending of event?
>>> (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
>>>
>> Currently in logServices( logEntry), we are not reporting the
>> "OriginOfCondition" property as per schema. I will check with Jason(
>> Who wrote the logService) and get back on this.
>>
>> BTW can you give me how this information is fetched in IBM way of
>> LogService implementation( D-Bus)? If you already ratified any such,
>> i think we can leverage. If not, We work together for solution.
>>
>>> * Any plan to add resource path in this journal message which
>>> tells that this is the path for which resource created event
>>> generated.
>>>
>> Same as above.
>>>
>>> * Would the path be "Redfish Path/ D-bus Path"?
>>>
>> As per Redfish specification, This should be "@odata.id" which means
>> it should be of resource uri and so we can't use d-bus path here for
>> OriginOfConditions.
>>>
>>> * Where the mapping would be done between D-busPath/Redfish
>>> Resource path?
>>>
>>>
>>> Cons: Every application have to make the change(use sd_journal_send).
>>> My thought is backend application should not be aware of the redfish terminlogy.
>>
>> Having separate process only for this mapping may not be good( No
>> different from maintaining that map inside bmcweb as there won't be
>> any other consumers). Ideal way is, that should be mapped while
>> logging logEntry's itself. But we are not doing it currently which
>> need to be re-looked. Give me some time, I will think and check with
>> other folks and get back.
>>
>>> *2.* Some application(bmcweb) would do the Inotify on the
>>> path(/var/log/redfish) and send the event once there is any activity
>>> on this file.
>>>
>>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
>>> per Intel design)
>>>
>>> Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
>> Initially i thought "ResourceType" based event filtering can be done
>> using minimal mapping( Using MessageID and args). But that will work
>> for minimal set. If the supported ResourceTypes grows, we will have
>> bigger challenges which i can sense it now. Anyway, Supported
>> Resources are completely implementation specific. If this value is
>> empty means, by default all event logs will be sent to subscribers.
>> This is what we can start with before supported ResourceTypes list
>> grows.
>>>>
>>>> As per my reading from below query, You are looking at d-bus match
>>>> rules and ResourceTypes mapping which is more specific to d-bus
>>>> event logging(IBM way of implementing event logging). reading it
>>>> from journal logs will give more information but that will impact
>>>> the performance to large extent. This might be one of the reason
>>>> why we (Intel) uses Redfish message ID while logging redfish events
>>>> logs to journal(You can refer design document for same at
>>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md).
>>>> In opinion, in your d-bus if you are using some kind of
>>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs
>>>> for all events and figure out the way to monitor the journal logs
>>>> without impacting the performance, that should be ok as long as
>>>> match filters are satisfied for Redfish EventService subscriptions
>>>> and supported Types(Again differs with implementation).
>>>>
>>>> Thanks,
>>>>
>>>> -Appu
>>>>
>>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>> ApparaRao.
>>>>>
>>>>> As you have shown interest in this feature and submitted the
>>>>> design document, do you have any opinion on this? Do you see any
>>>>> merit in using D-Bus match in bmcweb to create event logs for life
>>>>> cycle events? Please feel free to weigh in.
>>>>>
>>>>> Thanks,
>>>>> Rajes
>>>>>
>>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I am going through the bmcweb code for implementing Redfish
>>>>>> EventService based on the design document
>>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This
>>>>>> design is hooked to the journal based Redfish Event Logging. For
>>>>>> life cycle events(ResourceAdded, ResourceRemoved,
>>>>>> ResourceUpdated), using D-Bus match, bmcweb can create an event
>>>>>> log. This requires a JSON dictionary, comprising an array of
>>>>>> Redfish Resource Name and the D-Bus path. This approach works
>>>>>> only in case of one to one mapping of Redfish Resource Name and
>>>>>> the D-Bus path. For propertiesChanged events, if the Redfish
>>>>>> Resource property is not on the same D-Bus path or the Redfish
>>>>>> Resource property name is different from the D-Bus property name,
>>>>>> then an additional JSON dictionary to maintain this information
>>>>>> is required. With D-Bus match alone in the bmcweb, Redfish
>>>>>> EventService can't be fully supported. For the Message Registers
>>>>>> and the Resource Types that are supported, the relevant OpenBMC
>>>>>> application must create an event log in the journal using either
>>>>>> the phosphor::logging::entry or sd_journal_send() command.
>>>>>>
>>>>>> After realizing that with D-Bus match in the bmcweb alone can't
>>>>>> help to fully implement EventService, I prefer to avoid using
>>>>>> D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC
>>>>>> application that generated the event to create an event log in
>>>>>> the journal. Do you see any advantage of using combination of
>>>>>> D-Bus match in the bmcweb wherever it is possible and changes to
>>>>>> OpenBMC application in other cases to create an event log ?
>>>>>>
>>>>>> Your views are highly appreciated.
>>>>>>
>>>>>> Thanks,
>>>>>> Rajes
>>>>>
>>> Thanks
>>> Ratan
>>>
>>>
[-- Attachment #2: Type: text/html, Size: 15338 bytes --]
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-05-27 3:48 ` Puli, Apparao
@ 2020-05-27 11:50 ` RAJESWARAN THILLAIGOVINDAN
2020-05-27 18:58 ` Puli, Apparao
0 siblings, 1 reply; 44+ messages in thread
From: RAJESWARAN THILLAIGOVINDAN @ 2020-05-27 11:50 UTC (permalink / raw)
To: Puli, Apparao, Ratan Gupta, openbmc
[-- Attachment #1: Type: text/plain, Size: 10624 bytes --]
Apparao,
Thanks a lot for your suggestions. We lean towards using a dictionary to
map Redfish ResourceType to D-Bus objects path and vice versa and then
using D-Bus match to generate life cycle events. This way, the changes
are limited to bmcweb. The resource type and the origin resource URI
will be included in the event log. This requires change in the format of
event log file /var/log/redfish. I have commented the same in the server
sent event patch that you have uploaded. Kindly see if you can leave the
parsing of file to the OEMs. That way, the existing infrastructure can
be used by the OEMs to support other filtering mechanisms as defined in
the specification.
Thanks,
Rajes
On 27-05-2020 09:18, Puli, Apparao wrote:
>
> Hi Rajeswaran,
>
> Thanks for your mail. At the moment, I don't have plans to support
> the "ResourceTypes", "OriginResources" based filtering. Basically
> Intel uses file systems based redfish event logs( journalctl ->
> rsync-> filesystem) and doesn't use D-Bus mechanism like IBM uses. So
> I am not much familiar with D-Bus logging but some of the suggestions:
>
> 1) While logging redfish events over D-Bus itself, it can provide
> details on ResourceTypes and OriginResource URI/Path.
>
> This is ideal and most efficient way. Since we walked a walked
> long distance from start, Its hard to modify all the services to uses
> these 2 new input parameters while logging events( Requires change in
> almost all repo's)
>
> 2) For resourcesTypes: Can have mapping dictionary against all
> MessageId's. For OriginResources: I believe, event log over D-Bus is
> already holding the Path. If so, last 3/4 nodes of uri can be taken
> and mapped against the resources and that can be used in Event
> filtering. We did used same mechanism in case of telemetry while
> mapping MetricReportDefinitions to URI.
>
> Hope this helps.
>
> Thanks,
>
> -Appu
>
>
> On 5/26/2020 5:50 PM, RAJESWARAN THILLAIGOVINDAN wrote:
>>
>> Apparao,
>>
>> I see that you have uploaded a
>> patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441)
>> for supporting server sent events. This patch supports event
>> filtering based on registry prefix and/or messageId.
>>
>> I would like to know if you have any plan to support event filtering
>> based on resource type. If so, I would like to work together for a
>> better solution. Earlier, I have proposed a solution based on D-Bus
>> match using a dictionary. With that approach, the major challenge is
>> to map Redfish resource and properties to D-Bus object and properties
>> respectively. If D-Bus applications are modified to include
>> resource type and origin of condition in the event, then there is no
>> need for a map. But,that brings Redfish terminology to the
>> application. Also, this will become an overhead if an OEM is not
>> interested in Redfish event service.
>>
>> Thanks,
>> Rajes
>> On 25-02-2020 19:36, Puli, Apparao wrote:
>>>
>>> Hi Ratan,
>>>
>>> Comments in-line
>>>
>>> On 2/24/2020 12:07 PM, Ratan Gupta wrote:
>>>>
>>>> Hi Apparao,
>>>>
>>>> On 2/20/20 12:49 AM, Puli, Apparao wrote:
>>>>> Hi,
>>>>>
>>>>> I am sorry for late response as this mail is buried under and
>>>>> got struck back of my mind.
>>>>>
>>>>> As i did mentioned in EventService design document, EventLog
>>>>> Monitor service is not common across the organizations( Example:
>>>>> Intel uses the redfish event logs file and inotify mechanism for
>>>>> monitoring the event logs. Where as IBM uses d-bus event log
>>>>> mechanism and they can relay on match rules). That said challenges
>>>>> with ResourceType mapping will be different in both monitoring
>>>>> mechanisms. This is good point. Initially when i started
>>>>> EventService design, i thought we can have mapping in bmcweb for
>>>>> ResourceTypes with set of MessageID's for Logged events ( As per
>>>>> Intel design) but not sure that may become difficult when we
>>>>> expand supported ResourceTypes.
>>>>
>>>> If I am getting correctly, Here is the flow which Intel uses.
>>>>
>>>> 1. Individual repo have to push the logs using sd_journal_send
>>>> which will write to the file(/var/log/redfish) by using rsyslog
>>>> daemon
>>>>
>>>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
>>>> "REDFISH_MESSAGE_ID=%s",
>>>> "ResourceEvent.1.0.ResourceCreated",NULL);
>>>>
>>>> * How you would populate the "OriginOfCondition" during
>>>> sending of event?
>>>> (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
>>>>
>>> Currently in logServices( logEntry), we are not reporting the
>>> "OriginOfCondition" property as per schema. I will check with Jason(
>>> Who wrote the logService) and get back on this.
>>>
>>> BTW can you give me how this information is fetched in IBM way of
>>> LogService implementation( D-Bus)? If you already ratified any such,
>>> i think we can leverage. If not, We work together for solution.
>>>
>>>> * Any plan to add resource path in this journal message which
>>>> tells that this is the path for which resource created
>>>> event generated.
>>>>
>>> Same as above.
>>>>
>>>> * Would the path be "Redfish Path/ D-bus Path"?
>>>>
>>> As per Redfish specification, This should be "@odata.id" which means
>>> it should be of resource uri and so we can't use d-bus path here for
>>> OriginOfConditions.
>>>>
>>>> * Where the mapping would be done between D-busPath/Redfish
>>>> Resource path?
>>>>
>>>>
>>>> Cons: Every application have to make the change(use sd_journal_send).
>>>> My thought is backend application should not be aware of the redfish terminlogy.
>>>
>>> Having separate process only for this mapping may not be good( No
>>> different from maintaining that map inside bmcweb as there won't be
>>> any other consumers). Ideal way is, that should be mapped while
>>> logging logEntry's itself. But we are not doing it currently which
>>> need to be re-looked. Give me some time, I will think and check with
>>> other folks and get back.
>>>
>>>> *2.* Some application(bmcweb) would do the Inotify on the
>>>> path(/var/log/redfish) and send the event once there is any
>>>> activity on this file.
>>>>
>>>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
>>>> per Intel design)
>>>>
>>>> Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
>>> Initially i thought "ResourceType" based event filtering can be done
>>> using minimal mapping( Using MessageID and args). But that will work
>>> for minimal set. If the supported ResourceTypes grows, we will have
>>> bigger challenges which i can sense it now. Anyway, Supported
>>> Resources are completely implementation specific. If this value is
>>> empty means, by default all event logs will be sent to subscribers.
>>> This is what we can start with before supported ResourceTypes list
>>> grows.
>>>>>
>>>>> As per my reading from below query, You are looking at d-bus match
>>>>> rules and ResourceTypes mapping which is more specific to d-bus
>>>>> event logging(IBM way of implementing event logging). reading it
>>>>> from journal logs will give more information but that will impact
>>>>> the performance to large extent. This might be one of the reason
>>>>> why we (Intel) uses Redfish message ID while logging redfish
>>>>> events logs to journal(You can refer design document for same at
>>>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md).
>>>>> In opinion, in your d-bus if you are using some kind of
>>>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs
>>>>> for all events and figure out the way to monitor the journal logs
>>>>> without impacting the performance, that should be ok as long as
>>>>> match filters are satisfied for Redfish EventService subscriptions
>>>>> and supported Types(Again differs with implementation).
>>>>>
>>>>> Thanks,
>>>>>
>>>>> -Appu
>>>>>
>>>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>> ApparaRao.
>>>>>>
>>>>>> As you have shown interest in this feature and submitted the
>>>>>> design document, do you have any opinion on this? Do you see any
>>>>>> merit in using D-Bus match in bmcweb to create event logs for
>>>>>> life cycle events? Please feel free to weigh in.
>>>>>>
>>>>>> Thanks,
>>>>>> Rajes
>>>>>>
>>>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> I am going through the bmcweb code for implementing Redfish
>>>>>>> EventService based on the design document
>>>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This
>>>>>>> design is hooked to the journal based Redfish Event Logging. For
>>>>>>> life cycle events(ResourceAdded, ResourceRemoved,
>>>>>>> ResourceUpdated), using D-Bus match, bmcweb can create an event
>>>>>>> log. This requires a JSON dictionary, comprising an array of
>>>>>>> Redfish Resource Name and the D-Bus path. This approach works
>>>>>>> only in case of one to one mapping of Redfish Resource Name and
>>>>>>> the D-Bus path. For propertiesChanged events, if the Redfish
>>>>>>> Resource property is not on the same D-Bus path or the Redfish
>>>>>>> Resource property name is different from the D-Bus property
>>>>>>> name, then an additional JSON dictionary to maintain this
>>>>>>> information is required. With D-Bus match alone in the bmcweb,
>>>>>>> Redfish EventService can't be fully supported. For the Message
>>>>>>> Registers and the Resource Types that are supported, the
>>>>>>> relevant OpenBMC application must create an event log in the
>>>>>>> journal using either the phosphor::logging::entry or
>>>>>>> sd_journal_send() command.
>>>>>>>
>>>>>>> After realizing that with D-Bus match in the bmcweb alone can't
>>>>>>> help to fully implement EventService, I prefer to avoid using
>>>>>>> D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC
>>>>>>> application that generated the event to create an event log in
>>>>>>> the journal. Do you see any advantage of using combination of
>>>>>>> D-Bus match in the bmcweb wherever it is possible and changes to
>>>>>>> OpenBMC application in other cases to create an event log ?
>>>>>>>
>>>>>>> Your views are highly appreciated.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Rajes
>>>>>>
>>>> Thanks
>>>> Ratan
>>>>
>>>>
[-- Attachment #2: Type: text/html, Size: 17042 bytes --]
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-05-27 11:50 ` RAJESWARAN THILLAIGOVINDAN
@ 2020-05-27 18:58 ` Puli, Apparao
2020-05-28 13:26 ` Ratan Gupta
0 siblings, 1 reply; 44+ messages in thread
From: Puli, Apparao @ 2020-05-27 18:58 UTC (permalink / raw)
To: RAJESWARAN THILLAIGOVINDAN, Ratan Gupta, openbmc
[-- Attachment #1: Type: text/plain, Size: 11603 bytes --]
Rajes,
The dictionary to map Redfish resourceType and D-Bus object path( I
believe URI intern) is good. It will be great if you share design
document, if its done.
At the moment, redfish event logs file(/var/log/redfish) doesn't have
ResourceTypes and OriginResource fields. The Existing redfish event log
implementation(log service) also doesn't have support for that. You can
propose design change for same along with how it is used in event log
service. Same thing, can be adopted to EventService, once its agreed by
OEM's( I am thinking, it should go under new OEM specific compiler flag.
But that we can ratify later).
Thanks,
-Appu
On 5/27/2020 5:20 PM, RAJESWARAN THILLAIGOVINDAN wrote:
>
> Apparao,
>
> Thanks a lot for your suggestions. We lean towards using a dictionary
> to map Redfish ResourceType to D-Bus objects path and vice versa and
> then using D-Bus match to generate life cycle events. This way, the
> changes are limited to bmcweb. The resource type and the origin
> resource URI will be included in the event log. This requires change
> in the format of event log file /var/log/redfish. I have commented the
> same in the server sent event patch that you have uploaded. Kindly see
> if you can leave the parsing of file to the OEMs. That way, the
> existing infrastructure can be used by the OEMs to support other
> filtering mechanisms as defined in the specification.
>
> Thanks,
> Rajes
>
>
> On 27-05-2020 09:18, Puli, Apparao wrote:
>>
>> Hi Rajeswaran,
>>
>> Thanks for your mail. At the moment, I don't have plans to support
>> the "ResourceTypes", "OriginResources" based filtering. Basically
>> Intel uses file systems based redfish event logs( journalctl ->
>> rsync-> filesystem) and doesn't use D-Bus mechanism like IBM uses. So
>> I am not much familiar with D-Bus logging but some of the suggestions:
>>
>> 1) While logging redfish events over D-Bus itself, it can provide
>> details on ResourceTypes and OriginResource URI/Path.
>>
>> This is ideal and most efficient way. Since we walked a walked
>> long distance from start, Its hard to modify all the services to uses
>> these 2 new input parameters while logging events( Requires change in
>> almost all repo's)
>>
>> 2) For resourcesTypes: Can have mapping dictionary against all
>> MessageId's. For OriginResources: I believe, event log over D-Bus is
>> already holding the Path. If so, last 3/4 nodes of uri can be taken
>> and mapped against the resources and that can be used in Event
>> filtering. We did used same mechanism in case of telemetry while
>> mapping MetricReportDefinitions to URI.
>>
>> Hope this helps.
>>
>> Thanks,
>>
>> -Appu
>>
>>
>> On 5/26/2020 5:50 PM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>
>>> Apparao,
>>>
>>> I see that you have uploaded a
>>> patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441)
>>> for supporting server sent events. This patch supports event
>>> filtering based on registry prefix and/or messageId.
>>>
>>> I would like to know if you have any plan to support event filtering
>>> based on resource type. If so, I would like to work together for a
>>> better solution. Earlier, I have proposed a solution based on D-Bus
>>> match using a dictionary. With that approach, the major challenge is
>>> to map Redfish resource and properties to D-Bus object and
>>> properties respectively. If D-Bus applications are modified to
>>> include resource type and origin of condition in the event, then
>>> there is no need for a map. But,that brings Redfish terminology to
>>> the application. Also, this will become an overhead if an OEM is not
>>> interested in Redfish event service.
>>>
>>> Thanks,
>>> Rajes
>>> On 25-02-2020 19:36, Puli, Apparao wrote:
>>>>
>>>> Hi Ratan,
>>>>
>>>> Comments in-line
>>>>
>>>> On 2/24/2020 12:07 PM, Ratan Gupta wrote:
>>>>>
>>>>> Hi Apparao,
>>>>>
>>>>> On 2/20/20 12:49 AM, Puli, Apparao wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I am sorry for late response as this mail is buried under and
>>>>>> got struck back of my mind.
>>>>>>
>>>>>> As i did mentioned in EventService design document, EventLog
>>>>>> Monitor service is not common across the organizations( Example:
>>>>>> Intel uses the redfish event logs file and inotify mechanism for
>>>>>> monitoring the event logs. Where as IBM uses d-bus event log
>>>>>> mechanism and they can relay on match rules). That said
>>>>>> challenges with ResourceType mapping will be different in both
>>>>>> monitoring mechanisms. This is good point. Initially when i
>>>>>> started EventService design, i thought we can have mapping in
>>>>>> bmcweb for ResourceTypes with set of MessageID's for Logged
>>>>>> events ( As per Intel design) but not sure that may become
>>>>>> difficult when we expand supported ResourceTypes.
>>>>>
>>>>> If I am getting correctly, Here is the flow which Intel uses.
>>>>>
>>>>> 1. Individual repo have to push the logs using sd_journal_send
>>>>> which will write to the file(/var/log/redfish) by using
>>>>> rsyslog daemon
>>>>>
>>>>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
>>>>> "REDFISH_MESSAGE_ID=%s",
>>>>> "ResourceEvent.1.0.ResourceCreated",NULL);
>>>>>
>>>>> * How you would populate the "OriginOfCondition" during
>>>>> sending of event?
>>>>> (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
>>>>>
>>>> Currently in logServices( logEntry), we are not reporting the
>>>> "OriginOfCondition" property as per schema. I will check with
>>>> Jason( Who wrote the logService) and get back on this.
>>>>
>>>> BTW can you give me how this information is fetched in IBM way of
>>>> LogService implementation( D-Bus)? If you already ratified any
>>>> such, i think we can leverage. If not, We work together for solution.
>>>>
>>>>> * Any plan to add resource path in this journal message
>>>>> which tells that this is the path for which resource
>>>>> created event generated.
>>>>>
>>>> Same as above.
>>>>>
>>>>> * Would the path be "Redfish Path/ D-bus Path"?
>>>>>
>>>> As per Redfish specification, This should be "@odata.id" which
>>>> means it should be of resource uri and so we can't use d-bus path
>>>> here for OriginOfConditions.
>>>>>
>>>>> * Where the mapping would be done between D-busPath/Redfish
>>>>> Resource path?
>>>>>
>>>>>
>>>>> Cons: Every application have to make the change(use sd_journal_send).
>>>>> My thought is backend application should not be aware of the redfish terminlogy.
>>>>
>>>> Having separate process only for this mapping may not be good( No
>>>> different from maintaining that map inside bmcweb as there won't be
>>>> any other consumers). Ideal way is, that should be mapped while
>>>> logging logEntry's itself. But we are not doing it currently which
>>>> need to be re-looked. Give me some time, I will think and check
>>>> with other folks and get back.
>>>>
>>>>> *2.* Some application(bmcweb) would do the Inotify on the
>>>>> path(/var/log/redfish) and send the event once there is any
>>>>> activity on this file.
>>>>>
>>>>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
>>>>> per Intel design)
>>>>>
>>>>> Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
>>>> Initially i thought "ResourceType" based event filtering can be
>>>> done using minimal mapping( Using MessageID and args). But that
>>>> will work for minimal set. If the supported ResourceTypes grows, we
>>>> will have bigger challenges which i can sense it now. Anyway,
>>>> Supported Resources are completely implementation specific. If this
>>>> value is empty means, by default all event logs will be sent to
>>>> subscribers. This is what we can start with before supported
>>>> ResourceTypes list grows.
>>>>>>
>>>>>> As per my reading from below query, You are looking at d-bus
>>>>>> match rules and ResourceTypes mapping which is more specific to
>>>>>> d-bus event logging(IBM way of implementing event logging).
>>>>>> reading it from journal logs will give more information but that
>>>>>> will impact the performance to large extent. This might be one of
>>>>>> the reason why we (Intel) uses Redfish message ID while logging
>>>>>> redfish events logs to journal(You can refer design document for
>>>>>> same at
>>>>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md).
>>>>>> In opinion, in your d-bus if you are using some kind of
>>>>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs
>>>>>> for all events and figure out the way to monitor the journal logs
>>>>>> without impacting the performance, that should be ok as long as
>>>>>> match filters are satisfied for Redfish EventService
>>>>>> subscriptions and supported Types(Again differs with
>>>>>> implementation).
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> -Appu
>>>>>>
>>>>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>>> ApparaRao.
>>>>>>>
>>>>>>> As you have shown interest in this feature and submitted the
>>>>>>> design document, do you have any opinion on this? Do you see any
>>>>>>> merit in using D-Bus match in bmcweb to create event logs for
>>>>>>> life cycle events? Please feel free to weigh in.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Rajes
>>>>>>>
>>>>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I am going through the bmcweb code for implementing Redfish
>>>>>>>> EventService based on the design document
>>>>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This
>>>>>>>> design is hooked to the journal based Redfish Event Logging.
>>>>>>>> For life cycle events(ResourceAdded, ResourceRemoved,
>>>>>>>> ResourceUpdated), using D-Bus match, bmcweb can create an
>>>>>>>> event log. This requires a JSON dictionary, comprising an array
>>>>>>>> of Redfish Resource Name and the D-Bus path. This approach
>>>>>>>> works only in case of one to one mapping of Redfish Resource
>>>>>>>> Name and the D-Bus path. For propertiesChanged events, if the
>>>>>>>> Redfish Resource property is not on the same D-Bus path or the
>>>>>>>> Redfish Resource property name is different from the D-Bus
>>>>>>>> property name, then an additional JSON dictionary to maintain
>>>>>>>> this information is required. With D-Bus match alone in the
>>>>>>>> bmcweb, Redfish EventService can't be fully supported. For the
>>>>>>>> Message Registers and the Resource Types that are supported,
>>>>>>>> the relevant OpenBMC application must create an event log in
>>>>>>>> the journal using either the phosphor::logging::entry or
>>>>>>>> sd_journal_send() command.
>>>>>>>>
>>>>>>>> After realizing that with D-Bus match in the bmcweb alone can't
>>>>>>>> help to fully implement EventService, I prefer to avoid using
>>>>>>>> D-Bus match in bmcweb. Instead, I prefer to modify the OpenBMC
>>>>>>>> application that generated the event to create an event log in
>>>>>>>> the journal. Do you see any advantage of using combination of
>>>>>>>> D-Bus match in the bmcweb wherever it is possible and changes
>>>>>>>> to OpenBMC application in other cases to create an event log ?
>>>>>>>>
>>>>>>>> Your views are highly appreciated.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Rajes
>>>>>>>
>>>>> Thanks
>>>>> Ratan
>>>>>
>>>>>
[-- Attachment #2: Type: text/html, Size: 18852 bytes --]
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-05-27 18:58 ` Puli, Apparao
@ 2020-05-28 13:26 ` Ratan Gupta
2020-05-29 15:45 ` Redfish event log for new local user addition Puli, Apparao
0 siblings, 1 reply; 44+ messages in thread
From: Ratan Gupta @ 2020-05-28 13:26 UTC (permalink / raw)
To: openbmc, appa >> Puli, Apparao
[-- Attachment #1: Type: text/plain, Size: 12160 bytes --]
Hi Appu,
Can you get me an example say if I want an redfish event to be generated
once any local user gets added on the BMC.
What would be the steps to be done with the current design and
where(which repo)?
On 5/28/20 12:28 AM, Puli, Apparao wrote:
>
> Rajes,
>
> The dictionary to map Redfish resourceType and D-Bus object path( I
> believe URI intern) is good. It will be great if you share design
> document, if its done.
>
> At the moment, redfish event logs file(/var/log/redfish) doesn't have
> ResourceTypes and OriginResource fields. The Existing redfish event
> log implementation(log service) also doesn't have support for that.
> You can propose design change for same along with how it is used in
> event log service. Same thing, can be adopted to EventService, once
> its agreed by OEM's( I am thinking, it should go under new OEM
> specific compiler flag. But that we can ratify later).
>
> Thanks,
>
> -Appu
>
> On 5/27/2020 5:20 PM, RAJESWARAN THILLAIGOVINDAN wrote:
>>
>> Apparao,
>>
>> Thanks a lot for your suggestions. We lean towards using a dictionary
>> to map Redfish ResourceType to D-Bus objects path and vice versa and
>> then using D-Bus match to generate life cycle events. This way, the
>> changes are limited to bmcweb. The resource type and the origin
>> resource URI will be included in the event log. This requires change
>> in the format of event log file /var/log/redfish. I have commented
>> the same in the server sent event patch that you have uploaded.
>> Kindly see if you can leave the parsing of file to the OEMs. That
>> way, the existing infrastructure can be used by the OEMs to support
>> other filtering mechanisms as defined in the specification.
>>
>> Thanks,
>> Rajes
>>
>>
>> On 27-05-2020 09:18, Puli, Apparao wrote:
>>>
>>> Hi Rajeswaran,
>>>
>>> Thanks for your mail. At the moment, I don't have plans to support
>>> the "ResourceTypes", "OriginResources" based filtering. Basically
>>> Intel uses file systems based redfish event logs( journalctl ->
>>> rsync-> filesystem) and doesn't use D-Bus mechanism like IBM uses.
>>> So I am not much familiar with D-Bus logging but some of the
>>> suggestions:
>>>
>>> 1) While logging redfish events over D-Bus itself, it can provide
>>> details on ResourceTypes and OriginResource URI/Path.
>>>
>>> This is ideal and most efficient way. Since we walked a walked
>>> long distance from start, Its hard to modify all the services to
>>> uses these 2 new input parameters while logging events( Requires
>>> change in almost all repo's)
>>>
>>> 2) For resourcesTypes: Can have mapping dictionary against all
>>> MessageId's. For OriginResources: I believe, event log over D-Bus is
>>> already holding the Path. If so, last 3/4 nodes of uri can be taken
>>> and mapped against the resources and that can be used in Event
>>> filtering. We did used same mechanism in case of telemetry while
>>> mapping MetricReportDefinitions to URI.
>>>
>>> Hope this helps.
>>>
>>> Thanks,
>>>
>>> -Appu
>>>
>>>
>>> On 5/26/2020 5:50 PM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>
>>>> Apparao,
>>>>
>>>> I see that you have uploaded a
>>>> patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441)
>>>> for supporting server sent events. This patch supports event
>>>> filtering based on registry prefix and/or messageId.
>>>>
>>>> I would like to know if you have any plan to support event
>>>> filtering based on resource type. If so, I would like to work
>>>> together for a better solution. Earlier, I have proposed a solution
>>>> based on D-Bus match using a dictionary. With that approach, the
>>>> major challenge is to map Redfish resource and properties to D-Bus
>>>> object and properties respectively. If D-Bus applications are
>>>> modified to include resource type and origin of condition in the
>>>> event, then there is no need for a map. But,that brings Redfish
>>>> terminology to the application. Also, this will become an overhead
>>>> if an OEM is not interested in Redfish event service.
>>>>
>>>> Thanks,
>>>> Rajes
>>>> On 25-02-2020 19:36, Puli, Apparao wrote:
>>>>>
>>>>> Hi Ratan,
>>>>>
>>>>> Comments in-line
>>>>>
>>>>> On 2/24/2020 12:07 PM, Ratan Gupta wrote:
>>>>>>
>>>>>> Hi Apparao,
>>>>>>
>>>>>> On 2/20/20 12:49 AM, Puli, Apparao wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> I am sorry for late response as this mail is buried under and
>>>>>>> got struck back of my mind.
>>>>>>>
>>>>>>> As i did mentioned in EventService design document, EventLog
>>>>>>> Monitor service is not common across the organizations( Example:
>>>>>>> Intel uses the redfish event logs file and inotify mechanism for
>>>>>>> monitoring the event logs. Where as IBM uses d-bus event log
>>>>>>> mechanism and they can relay on match rules). That said
>>>>>>> challenges with ResourceType mapping will be different in both
>>>>>>> monitoring mechanisms. This is good point. Initially when i
>>>>>>> started EventService design, i thought we can have mapping in
>>>>>>> bmcweb for ResourceTypes with set of MessageID's for Logged
>>>>>>> events ( As per Intel design) but not sure that may become
>>>>>>> difficult when we expand supported ResourceTypes.
>>>>>>
>>>>>> If I am getting correctly, Here is the flow which Intel uses.
>>>>>>
>>>>>> 1. Individual repo have to push the logs using sd_journal_send
>>>>>> which will write to the file(/var/log/redfish) by using
>>>>>> rsyslog daemon
>>>>>>
>>>>>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
>>>>>> "REDFISH_MESSAGE_ID=%s",
>>>>>> "ResourceEvent.1.0.ResourceCreated",NULL);
>>>>>>
>>>>>> * How you would populate the "OriginOfCondition" during
>>>>>> sending of event?
>>>>>> (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
>>>>>>
>>>>> Currently in logServices( logEntry), we are not reporting the
>>>>> "OriginOfCondition" property as per schema. I will check with
>>>>> Jason( Who wrote the logService) and get back on this.
>>>>>
>>>>> BTW can you give me how this information is fetched in IBM way of
>>>>> LogService implementation( D-Bus)? If you already ratified any
>>>>> such, i think we can leverage. If not, We work together for
>>>>> solution.
>>>>>
>>>>>> * Any plan to add resource path in this journal message
>>>>>> which tells that this is the path for which resource
>>>>>> created event generated.
>>>>>>
>>>>> Same as above.
>>>>>>
>>>>>> * Would the path be "Redfish Path/ D-bus Path"?
>>>>>>
>>>>> As per Redfish specification, This should be "@odata.id" which
>>>>> means it should be of resource uri and so we can't use d-bus path
>>>>> here for OriginOfConditions.
>>>>>>
>>>>>> * Where the mapping would be done between D-busPath/Redfish
>>>>>> Resource path?
>>>>>>
>>>>>>
>>>>>> Cons: Every application have to make the change(use sd_journal_send).
>>>>>> My thought is backend application should not be aware of the redfish terminlogy.
>>>>>
>>>>> Having separate process only for this mapping may not be good( No
>>>>> different from maintaining that map inside bmcweb as there won't
>>>>> be any other consumers). Ideal way is, that should be mapped while
>>>>> logging logEntry's itself. But we are not doing it currently which
>>>>> need to be re-looked. Give me some time, I will think and check
>>>>> with other folks and get back.
>>>>>
>>>>>> *2.* Some application(bmcweb) would do the Inotify on the
>>>>>> path(/var/log/redfish) and send the event once there is any
>>>>>> activity on this file.
>>>>>>
>>>>>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
>>>>>> per Intel design)
>>>>>>
>>>>>> Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
>>>>> Initially i thought "ResourceType" based event filtering can be
>>>>> done using minimal mapping( Using MessageID and args). But that
>>>>> will work for minimal set. If the supported ResourceTypes grows,
>>>>> we will have bigger challenges which i can sense it now. Anyway,
>>>>> Supported Resources are completely implementation specific. If
>>>>> this value is empty means, by default all event logs will be sent
>>>>> to subscribers. This is what we can start with before supported
>>>>> ResourceTypes list grows.
>>>>>>>
>>>>>>> As per my reading from below query, You are looking at d-bus
>>>>>>> match rules and ResourceTypes mapping which is more specific to
>>>>>>> d-bus event logging(IBM way of implementing event logging).
>>>>>>> reading it from journal logs will give more information but that
>>>>>>> will impact the performance to large extent. This might be one
>>>>>>> of the reason why we (Intel) uses Redfish message ID while
>>>>>>> logging redfish events logs to journal(You can refer design
>>>>>>> document for same at
>>>>>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md).
>>>>>>> In opinion, in your d-bus if you are using some kind of
>>>>>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal logs
>>>>>>> for all events and figure out the way to monitor the journal
>>>>>>> logs without impacting the performance, that should be ok as
>>>>>>> long as match filters are satisfied for Redfish EventService
>>>>>>> subscriptions and supported Types(Again differs with
>>>>>>> implementation).
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> -Appu
>>>>>>>
>>>>>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>>>> ApparaRao.
>>>>>>>>
>>>>>>>> As you have shown interest in this feature and submitted the
>>>>>>>> design document, do you have any opinion on this? Do you see
>>>>>>>> any merit in using D-Bus match in bmcweb to create event logs
>>>>>>>> for life cycle events? Please feel free to weigh in.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Rajes
>>>>>>>>
>>>>>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I am going through the bmcweb code for implementing Redfish
>>>>>>>>> EventService based on the design document
>>>>>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749.
>>>>>>>>> This design is hooked to the journal based Redfish Event
>>>>>>>>> Logging. For life cycle events(ResourceAdded, ResourceRemoved,
>>>>>>>>> ResourceUpdated), using D-Bus match, bmcweb can create an
>>>>>>>>> event log. This requires a JSON dictionary, comprising an
>>>>>>>>> array of Redfish Resource Name and the D-Bus path. This
>>>>>>>>> approach works only in case of one to one mapping of Redfish
>>>>>>>>> Resource Name and the D-Bus path. For propertiesChanged
>>>>>>>>> events, if the Redfish Resource property is not on the same
>>>>>>>>> D-Bus path or the Redfish Resource property name is different
>>>>>>>>> from the D-Bus property name, then an additional JSON
>>>>>>>>> dictionary to maintain this information is required. With
>>>>>>>>> D-Bus match alone in the bmcweb, Redfish EventService can't be
>>>>>>>>> fully supported. For the Message Registers and the Resource
>>>>>>>>> Types that are supported, the relevant OpenBMC application
>>>>>>>>> must create an event log in the journal using either the
>>>>>>>>> phosphor::logging::entry or sd_journal_send() command.
>>>>>>>>>
>>>>>>>>> After realizing that with D-Bus match in the bmcweb alone
>>>>>>>>> can't help to fully implement EventService, I prefer to avoid
>>>>>>>>> using D-Bus match in bmcweb. Instead, I prefer to modify the
>>>>>>>>> OpenBMC application that generated the event to create an
>>>>>>>>> event log in the journal. Do you see any advantage of using
>>>>>>>>> combination of D-Bus match in the bmcweb wherever it is
>>>>>>>>> possible and changes to OpenBMC application in other cases to
>>>>>>>>> create an event log ?
>>>>>>>>>
>>>>>>>>> Your views are highly appreciated.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Rajes
>>>>>>>>
>>>>>> Thanks
>>>>>> Ratan
>>>>>>
>>>>>>
Ratan
[-- Attachment #2: Type: text/html, Size: 20206 bytes --]
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish event log for new local user addition
2020-05-28 13:26 ` Ratan Gupta
@ 2020-05-29 15:45 ` Puli, Apparao
2020-06-02 6:30 ` Ratan Gupta
0 siblings, 1 reply; 44+ messages in thread
From: Puli, Apparao @ 2020-05-29 15:45 UTC (permalink / raw)
To: Ratan Gupta, openbmc
[-- Attachment #1: Type: text/plain, Size: 13233 bytes --]
[Subject changed to reflect actual context]
Hi Ratan,
With current OpenBMC, I don't any event log getting added for new
local user creation. To implement that below are two place we need to
add support:
1) Define an new message retry ID for user addition(say UserAdded) in
bmcweb.
https://github.com/openbmc/bmcweb/blob/master/redfish-core/include/registries/openbmc_message_registry.hpp
2) Add a event log during new user addition inside
phospor-user-manager(createUser function).
https://github.com/openbmc/phosphor-user-manager/blob/master/user_mgr.cpp#L336
While you are adding event log for new local user creation, Please
consider other case too( Like Delete, Update, Rename etc..)
Thanks,
-Appu
On 5/28/2020 6:56 PM, Ratan Gupta wrote:
>
> Hi Appu,
>
> Can you get me an example say if I want an redfish event to be
> generated once any local user gets added on the BMC.
>
> What would be the steps to be done with the current design and
> where(which repo)?
>
>
> On 5/28/20 12:28 AM, Puli, Apparao wrote:
>>
>> Rajes,
>>
>> The dictionary to map Redfish resourceType and D-Bus object path(
>> I believe URI intern) is good. It will be great if you share design
>> document, if its done.
>>
>> At the moment, redfish event logs file(/var/log/redfish) doesn't have
>> ResourceTypes and OriginResource fields. The Existing redfish event
>> log implementation(log service) also doesn't have support for that.
>> You can propose design change for same along with how it is used in
>> event log service. Same thing, can be adopted to EventService, once
>> its agreed by OEM's( I am thinking, it should go under new OEM
>> specific compiler flag. But that we can ratify later).
>>
>> Thanks,
>>
>> -Appu
>>
>> On 5/27/2020 5:20 PM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>
>>> Apparao,
>>>
>>> Thanks a lot for your suggestions. We lean towards using a
>>> dictionary to map Redfish ResourceType to D-Bus objects path and
>>> vice versa and then using D-Bus match to generate life cycle events.
>>> This way, the changes are limited to bmcweb. The resource type and
>>> the origin resource URI will be included in the event log. This
>>> requires change in the format of event log file /var/log/redfish. I
>>> have commented the same in the server sent event patch that you have
>>> uploaded. Kindly see if you can leave the parsing of file to the
>>> OEMs. That way, the existing infrastructure can be used by the OEMs
>>> to support other filtering mechanisms as defined in the specification.
>>>
>>> Thanks,
>>> Rajes
>>>
>>>
>>> On 27-05-2020 09:18, Puli, Apparao wrote:
>>>>
>>>> Hi Rajeswaran,
>>>>
>>>> Thanks for your mail. At the moment, I don't have plans to
>>>> support the "ResourceTypes", "OriginResources" based filtering.
>>>> Basically Intel uses file systems based redfish event logs(
>>>> journalctl -> rsync-> filesystem) and doesn't use D-Bus mechanism
>>>> like IBM uses. So I am not much familiar with D-Bus logging but
>>>> some of the suggestions:
>>>>
>>>> 1) While logging redfish events over D-Bus itself, it can provide
>>>> details on ResourceTypes and OriginResource URI/Path.
>>>>
>>>> This is ideal and most efficient way. Since we walked a
>>>> walked long distance from start, Its hard to modify all the
>>>> services to uses these 2 new input parameters while logging events(
>>>> Requires change in almost all repo's)
>>>>
>>>> 2) For resourcesTypes: Can have mapping dictionary against all
>>>> MessageId's. For OriginResources: I believe, event log over D-Bus
>>>> is already holding the Path. If so, last 3/4 nodes of uri can be
>>>> taken and mapped against the resources and that can be used in
>>>> Event filtering. We did used same mechanism in case of telemetry
>>>> while mapping MetricReportDefinitions to URI.
>>>>
>>>> Hope this helps.
>>>>
>>>> Thanks,
>>>>
>>>> -Appu
>>>>
>>>>
>>>> On 5/26/2020 5:50 PM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>
>>>>> Apparao,
>>>>>
>>>>> I see that you have uploaded a
>>>>> patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441)
>>>>> for supporting server sent events. This patch supports event
>>>>> filtering based on registry prefix and/or messageId.
>>>>>
>>>>> I would like to know if you have any plan to support event
>>>>> filtering based on resource type. If so, I would like to work
>>>>> together for a better solution. Earlier, I have proposed a
>>>>> solution based on D-Bus match using a dictionary. With that
>>>>> approach, the major challenge is to map Redfish resource and
>>>>> properties to D-Bus object and properties respectively. If D-Bus
>>>>> applications are modified to include resource type and origin of
>>>>> condition in the event, then there is no need for a map. But,that
>>>>> brings Redfish terminology to the application. Also, this will
>>>>> become an overhead if an OEM is not interested in Redfish event
>>>>> service.
>>>>>
>>>>> Thanks,
>>>>> Rajes
>>>>> On 25-02-2020 19:36, Puli, Apparao wrote:
>>>>>>
>>>>>> Hi Ratan,
>>>>>>
>>>>>> Comments in-line
>>>>>>
>>>>>> On 2/24/2020 12:07 PM, Ratan Gupta wrote:
>>>>>>>
>>>>>>> Hi Apparao,
>>>>>>>
>>>>>>> On 2/20/20 12:49 AM, Puli, Apparao wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I am sorry for late response as this mail is buried under and
>>>>>>>> got struck back of my mind.
>>>>>>>>
>>>>>>>> As i did mentioned in EventService design document, EventLog
>>>>>>>> Monitor service is not common across the organizations(
>>>>>>>> Example: Intel uses the redfish event logs file and inotify
>>>>>>>> mechanism for monitoring the event logs. Where as IBM uses
>>>>>>>> d-bus event log mechanism and they can relay on match rules).
>>>>>>>> That said challenges with ResourceType mapping will be
>>>>>>>> different in both monitoring mechanisms. This is good point.
>>>>>>>> Initially when i started EventService design, i thought we can
>>>>>>>> have mapping in bmcweb for ResourceTypes with set of
>>>>>>>> MessageID's for Logged events ( As per Intel design) but not
>>>>>>>> sure that may become difficult when we expand supported
>>>>>>>> ResourceTypes.
>>>>>>>
>>>>>>> If I am getting correctly, Here is the flow which Intel uses.
>>>>>>>
>>>>>>> 1. Individual repo have to push the logs using sd_journal_send
>>>>>>> which will write to the file(/var/log/redfish) by using
>>>>>>> rsyslog daemon
>>>>>>>
>>>>>>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
>>>>>>> "REDFISH_MESSAGE_ID=%s",
>>>>>>> "ResourceEvent.1.0.ResourceCreated",NULL);
>>>>>>>
>>>>>>> * How you would populate the "OriginOfCondition" during
>>>>>>> sending of event?
>>>>>>> (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
>>>>>>>
>>>>>> Currently in logServices( logEntry), we are not reporting the
>>>>>> "OriginOfCondition" property as per schema. I will check with
>>>>>> Jason( Who wrote the logService) and get back on this.
>>>>>>
>>>>>> BTW can you give me how this information is fetched in IBM way of
>>>>>> LogService implementation( D-Bus)? If you already ratified any
>>>>>> such, i think we can leverage. If not, We work together for
>>>>>> solution.
>>>>>>
>>>>>>> * Any plan to add resource path in this journal message
>>>>>>> which tells that this is the path for which resource
>>>>>>> created event generated.
>>>>>>>
>>>>>> Same as above.
>>>>>>>
>>>>>>> * Would the path be "Redfish Path/ D-bus Path"?
>>>>>>>
>>>>>> As per Redfish specification, This should be "@odata.id" which
>>>>>> means it should be of resource uri and so we can't use d-bus path
>>>>>> here for OriginOfConditions.
>>>>>>>
>>>>>>> * Where the mapping would be done between
>>>>>>> D-busPath/Redfish Resource path?
>>>>>>>
>>>>>>>
>>>>>>> Cons: Every application have to make the change(use sd_journal_send).
>>>>>>> My thought is backend application should not be aware of the redfish terminlogy.
>>>>>>
>>>>>> Having separate process only for this mapping may not be good( No
>>>>>> different from maintaining that map inside bmcweb as there won't
>>>>>> be any other consumers). Ideal way is, that should be mapped
>>>>>> while logging logEntry's itself. But we are not doing it
>>>>>> currently which need to be re-looked. Give me some time, I will
>>>>>> think and check with other folks and get back.
>>>>>>
>>>>>>> *2.* Some application(bmcweb) would do the Inotify on the
>>>>>>> path(/var/log/redfish) and send the event once there is any
>>>>>>> activity on this file.
>>>>>>>
>>>>>>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
>>>>>>> per Intel design)
>>>>>>>
>>>>>>> Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
>>>>>> Initially i thought "ResourceType" based event filtering can be
>>>>>> done using minimal mapping( Using MessageID and args). But that
>>>>>> will work for minimal set. If the supported ResourceTypes grows,
>>>>>> we will have bigger challenges which i can sense it now. Anyway,
>>>>>> Supported Resources are completely implementation specific. If
>>>>>> this value is empty means, by default all event logs will be sent
>>>>>> to subscribers. This is what we can start with before supported
>>>>>> ResourceTypes list grows.
>>>>>>>>
>>>>>>>> As per my reading from below query, You are looking at d-bus
>>>>>>>> match rules and ResourceTypes mapping which is more specific to
>>>>>>>> d-bus event logging(IBM way of implementing event logging).
>>>>>>>> reading it from journal logs will give more information but
>>>>>>>> that will impact the performance to large extent. This might be
>>>>>>>> one of the reason why we (Intel) uses Redfish message ID while
>>>>>>>> logging redfish events logs to journal(You can refer design
>>>>>>>> document for same at
>>>>>>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md).
>>>>>>>> In opinion, in your d-bus if you are using some kind of
>>>>>>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal
>>>>>>>> logs for all events and figure out the way to monitor the
>>>>>>>> journal logs without impacting the performance, that should be
>>>>>>>> ok as long as match filters are satisfied for Redfish
>>>>>>>> EventService subscriptions and supported Types(Again differs
>>>>>>>> with implementation).
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> -Appu
>>>>>>>>
>>>>>>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>>>>> ApparaRao.
>>>>>>>>>
>>>>>>>>> As you have shown interest in this feature and submitted the
>>>>>>>>> design document, do you have any opinion on this? Do you see
>>>>>>>>> any merit in using D-Bus match in bmcweb to create event logs
>>>>>>>>> for life cycle events? Please feel free to weigh in.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Rajes
>>>>>>>>>
>>>>>>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I am going through the bmcweb code for implementing Redfish
>>>>>>>>>> EventService based on the design document
>>>>>>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749.
>>>>>>>>>> This design is hooked to the journal based Redfish Event
>>>>>>>>>> Logging. For life cycle events(ResourceAdded,
>>>>>>>>>> ResourceRemoved, ResourceUpdated), using D-Bus match, bmcweb
>>>>>>>>>> can create an event log. This requires a JSON dictionary,
>>>>>>>>>> comprising an array of Redfish Resource Name and the D-Bus
>>>>>>>>>> path. This approach works only in case of one to one mapping
>>>>>>>>>> of Redfish Resource Name and the D-Bus path. For
>>>>>>>>>> propertiesChanged events, if the Redfish Resource property is
>>>>>>>>>> not on the same D-Bus path or the Redfish Resource property
>>>>>>>>>> name is different from the D-Bus property name, then an
>>>>>>>>>> additional JSON dictionary to maintain this information is
>>>>>>>>>> required. With D-Bus match alone in the bmcweb, Redfish
>>>>>>>>>> EventService can't be fully supported. For the Message
>>>>>>>>>> Registers and the Resource Types that are supported, the
>>>>>>>>>> relevant OpenBMC application must create an event log in the
>>>>>>>>>> journal using either the phosphor::logging::entry or
>>>>>>>>>> sd_journal_send() command.
>>>>>>>>>>
>>>>>>>>>> After realizing that with D-Bus match in the bmcweb alone
>>>>>>>>>> can't help to fully implement EventService, I prefer to avoid
>>>>>>>>>> using D-Bus match in bmcweb. Instead, I prefer to modify the
>>>>>>>>>> OpenBMC application that generated the event to create an
>>>>>>>>>> event log in the journal. Do you see any advantage of using
>>>>>>>>>> combination of D-Bus match in the bmcweb wherever it is
>>>>>>>>>> possible and changes to OpenBMC application in other cases to
>>>>>>>>>> create an event log ?
>>>>>>>>>>
>>>>>>>>>> Your views are highly appreciated.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Rajes
>>>>>>>>>
>>>>>>> Thanks
>>>>>>> Ratan
>>>>>>>
>>>>>>>
> Ratan
[-- Attachment #2: Type: text/html, Size: 22528 bytes --]
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish event log for new local user addition
2020-05-29 15:45 ` Redfish event log for new local user addition Puli, Apparao
@ 2020-06-02 6:30 ` Ratan Gupta
0 siblings, 0 replies; 44+ messages in thread
From: Ratan Gupta @ 2020-06-02 6:30 UTC (permalink / raw)
To: Puli, Apparao, openbmc
[-- Attachment #1: Type: text/plain, Size: 13972 bytes --]
Hi Appu,
Thanks for responding.
As the Impact would be on all the openbmc repo where we need to add the
event log for any properties change or interface added.
I feel it is kind of overhead and not realistic as each and every repo
need to know about redfish terminology eg : Redfish Path, Redfish
property, Redfish Registry Prefix etc.
I feel it is a big work.
Ratan
On 5/29/20 9:15 PM, Puli, Apparao wrote:
>
> [Subject changed to reflect actual context]
>
> Hi Ratan,
>
> With current OpenBMC, I don't any event log getting added for new
> local user creation. To implement that below are two place we need to
> add support:
>
> 1) Define an new message retry ID for user addition(say UserAdded) in
> bmcweb.
>
> https://github.com/openbmc/bmcweb/blob/master/redfish-core/include/registries/openbmc_message_registry.hpp
>
> 2) Add a event log during new user addition inside
> phospor-user-manager(createUser function).
>
> https://github.com/openbmc/phosphor-user-manager/blob/master/user_mgr.cpp#L336
>
>
> While you are adding event log for new local user creation, Please
> consider other case too( Like Delete, Update, Rename etc..)
>
> Thanks,
>
> -Appu
>
>
> On 5/28/2020 6:56 PM, Ratan Gupta wrote:
>>
>> Hi Appu,
>>
>> Can you get me an example say if I want an redfish event to be
>> generated once any local user gets added on the BMC.
>>
>> What would be the steps to be done with the current design and
>> where(which repo)?
>>
>>
>> On 5/28/20 12:28 AM, Puli, Apparao wrote:
>>>
>>> Rajes,
>>>
>>> The dictionary to map Redfish resourceType and D-Bus object path(
>>> I believe URI intern) is good. It will be great if you share design
>>> document, if its done.
>>>
>>> At the moment, redfish event logs file(/var/log/redfish) doesn't
>>> have ResourceTypes and OriginResource fields. The Existing redfish
>>> event log implementation(log service) also doesn't have support for
>>> that. You can propose design change for same along with how it is
>>> used in event log service. Same thing, can be adopted to
>>> EventService, once its agreed by OEM's( I am thinking, it should go
>>> under new OEM specific compiler flag. But that we can ratify later).
>>>
>>> Thanks,
>>>
>>> -Appu
>>>
>>> On 5/27/2020 5:20 PM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>
>>>> Apparao,
>>>>
>>>> Thanks a lot for your suggestions. We lean towards using a
>>>> dictionary to map Redfish ResourceType to D-Bus objects path and
>>>> vice versa and then using D-Bus match to generate life cycle
>>>> events. This way, the changes are limited to bmcweb. The resource
>>>> type and the origin resource URI will be included in the event log.
>>>> This requires change in the format of event log file
>>>> /var/log/redfish. I have commented the same in the server sent
>>>> event patch that you have uploaded. Kindly see if you can leave the
>>>> parsing of file to the OEMs. That way, the existing infrastructure
>>>> can be used by the OEMs to support other filtering mechanisms as
>>>> defined in the specification.
>>>>
>>>> Thanks,
>>>> Rajes
>>>>
>>>>
>>>> On 27-05-2020 09:18, Puli, Apparao wrote:
>>>>>
>>>>> Hi Rajeswaran,
>>>>>
>>>>> Thanks for your mail. At the moment, I don't have plans to
>>>>> support the "ResourceTypes", "OriginResources" based filtering.
>>>>> Basically Intel uses file systems based redfish event logs(
>>>>> journalctl -> rsync-> filesystem) and doesn't use D-Bus mechanism
>>>>> like IBM uses. So I am not much familiar with D-Bus logging but
>>>>> some of the suggestions:
>>>>>
>>>>> 1) While logging redfish events over D-Bus itself, it can
>>>>> provide details on ResourceTypes and OriginResource URI/Path.
>>>>>
>>>>> This is ideal and most efficient way. Since we walked a
>>>>> walked long distance from start, Its hard to modify all the
>>>>> services to uses these 2 new input parameters while logging
>>>>> events( Requires change in almost all repo's)
>>>>>
>>>>> 2) For resourcesTypes: Can have mapping dictionary against all
>>>>> MessageId's. For OriginResources: I believe, event log over D-Bus
>>>>> is already holding the Path. If so, last 3/4 nodes of uri can be
>>>>> taken and mapped against the resources and that can be used in
>>>>> Event filtering. We did used same mechanism in case of telemetry
>>>>> while mapping MetricReportDefinitions to URI.
>>>>>
>>>>> Hope this helps.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> -Appu
>>>>>
>>>>>
>>>>> On 5/26/2020 5:50 PM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>>
>>>>>> Apparao,
>>>>>>
>>>>>> I see that you have uploaded a
>>>>>> patch(https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/32441)
>>>>>> for supporting server sent events. This patch supports event
>>>>>> filtering based on registry prefix and/or messageId.
>>>>>>
>>>>>> I would like to know if you have any plan to support event
>>>>>> filtering based on resource type. If so, I would like to work
>>>>>> together for a better solution. Earlier, I have proposed a
>>>>>> solution based on D-Bus match using a dictionary. With that
>>>>>> approach, the major challenge is to map Redfish resource and
>>>>>> properties to D-Bus object and properties respectively. If
>>>>>> D-Bus applications are modified to include resource type and
>>>>>> origin of condition in the event, then there is no need for a
>>>>>> map. But,that brings Redfish terminology to the application.
>>>>>> Also, this will become an overhead if an OEM is not interested in
>>>>>> Redfish event service.
>>>>>>
>>>>>> Thanks,
>>>>>> Rajes
>>>>>> On 25-02-2020 19:36, Puli, Apparao wrote:
>>>>>>>
>>>>>>> Hi Ratan,
>>>>>>>
>>>>>>> Comments in-line
>>>>>>>
>>>>>>> On 2/24/2020 12:07 PM, Ratan Gupta wrote:
>>>>>>>>
>>>>>>>> Hi Apparao,
>>>>>>>>
>>>>>>>> On 2/20/20 12:49 AM, Puli, Apparao wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I am sorry for late response as this mail is buried under
>>>>>>>>> and got struck back of my mind.
>>>>>>>>>
>>>>>>>>> As i did mentioned in EventService design document, EventLog
>>>>>>>>> Monitor service is not common across the organizations(
>>>>>>>>> Example: Intel uses the redfish event logs file and inotify
>>>>>>>>> mechanism for monitoring the event logs. Where as IBM uses
>>>>>>>>> d-bus event log mechanism and they can relay on match rules).
>>>>>>>>> That said challenges with ResourceType mapping will be
>>>>>>>>> different in both monitoring mechanisms. This is good point.
>>>>>>>>> Initially when i started EventService design, i thought we can
>>>>>>>>> have mapping in bmcweb for ResourceTypes with set of
>>>>>>>>> MessageID's for Logged events ( As per Intel design) but not
>>>>>>>>> sure that may become difficult when we expand supported
>>>>>>>>> ResourceTypes.
>>>>>>>>
>>>>>>>> If I am getting correctly, Here is the flow which Intel uses.
>>>>>>>>
>>>>>>>> 1. Individual repo have to push the logs using sd_journal_send
>>>>>>>> which will write to the file(/var/log/redfish) by using
>>>>>>>> rsyslog daemon
>>>>>>>>
>>>>>>>> sd_journal_send("MESSAGE=%s","journal text","PRIORITY=%i", <LOG_LEVEL>,
>>>>>>>> "REDFISH_MESSAGE_ID=%s",
>>>>>>>> "ResourceEvent.1.0.ResourceCreated",NULL);
>>>>>>>>
>>>>>>>> * How you would populate the "OriginOfCondition" during
>>>>>>>> sending of event?
>>>>>>>> (https://redfish.dmtf.org/schemas/v1/Event.v1_4_1.json)
>>>>>>>>
>>>>>>> Currently in logServices( logEntry), we are not reporting the
>>>>>>> "OriginOfCondition" property as per schema. I will check with
>>>>>>> Jason( Who wrote the logService) and get back on this.
>>>>>>>
>>>>>>> BTW can you give me how this information is fetched in IBM way
>>>>>>> of LogService implementation( D-Bus)? If you already ratified
>>>>>>> any such, i think we can leverage. If not, We work together for
>>>>>>> solution.
>>>>>>>
>>>>>>>> * Any plan to add resource path in this journal message
>>>>>>>> which tells that this is the path for which resource
>>>>>>>> created event generated.
>>>>>>>>
>>>>>>> Same as above.
>>>>>>>>
>>>>>>>> * Would the path be "Redfish Path/ D-bus Path"?
>>>>>>>>
>>>>>>> As per Redfish specification, This should be "@odata.id" which
>>>>>>> means it should be of resource uri and so we can't use d-bus
>>>>>>> path here for OriginOfConditions.
>>>>>>>>
>>>>>>>> * Where the mapping would be done between
>>>>>>>> D-busPath/Redfish Resource path?
>>>>>>>>
>>>>>>>>
>>>>>>>> Cons: Every application have to make the change(use sd_journal_send).
>>>>>>>> My thought is backend application should not be aware of the redfish terminlogy.
>>>>>>>
>>>>>>> Having separate process only for this mapping may not be good(
>>>>>>> No different from maintaining that map inside bmcweb as there
>>>>>>> won't be any other consumers). Ideal way is, that should be
>>>>>>> mapped while logging logEntry's itself. But we are not doing it
>>>>>>> currently which need to be re-looked. Give me some time, I will
>>>>>>> think and check with other folks and get back.
>>>>>>>
>>>>>>>> *2.* Some application(bmcweb) would do the Inotify on the
>>>>>>>> path(/var/log/redfish) and send the event once there is any
>>>>>>>> activity on this file.
>>>>>>>>
>>>>>>>> > I thought we can have mapping in bmcweb for ResourceTypes with set of MessageID's for Logged events ( As
>>>>>>>> per Intel design)
>>>>>>>>
>>>>>>>> Can you explain more here. What is your plan? How you would do the Resource Type based event filtering?REDFISH_MESSAGE_ID is different than the resource type.
>>>>>>> Initially i thought "ResourceType" based event filtering can be
>>>>>>> done using minimal mapping( Using MessageID and args). But that
>>>>>>> will work for minimal set. If the supported ResourceTypes grows,
>>>>>>> we will have bigger challenges which i can sense it now. Anyway,
>>>>>>> Supported Resources are completely implementation specific. If
>>>>>>> this value is empty means, by default all event logs will be
>>>>>>> sent to subscribers. This is what we can start with before
>>>>>>> supported ResourceTypes list grows.
>>>>>>>>>
>>>>>>>>> As per my reading from below query, You are looking at d-bus
>>>>>>>>> match rules and ResourceTypes mapping which is more specific
>>>>>>>>> to d-bus event logging(IBM way of implementing event logging).
>>>>>>>>> reading it from journal logs will give more information but
>>>>>>>>> that will impact the performance to large extent. This might
>>>>>>>>> be one of the reason why we (Intel) uses Redfish message ID
>>>>>>>>> while logging redfish events logs to journal(You can refer
>>>>>>>>> design document for same at
>>>>>>>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md).
>>>>>>>>> In opinion, in your d-bus if you are using some kind of
>>>>>>>>> filter(Example REDFISH_MESSAGE_ID) while logging in journal
>>>>>>>>> logs for all events and figure out the way to monitor the
>>>>>>>>> journal logs without impacting the performance, that should be
>>>>>>>>> ok as long as match filters are satisfied for Redfish
>>>>>>>>> EventService subscriptions and supported Types(Again differs
>>>>>>>>> with implementation).
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> -Appu
>>>>>>>>>
>>>>>>>>> On 2/10/2020 1:52 AM, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>>>>>> ApparaRao.
>>>>>>>>>>
>>>>>>>>>> As you have shown interest in this feature and submitted the
>>>>>>>>>> design document, do you have any opinion on this? Do you see
>>>>>>>>>> any merit in using D-Bus match in bmcweb to create event logs
>>>>>>>>>> for life cycle events? Please feel free to weigh in.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Rajes
>>>>>>>>>>
>>>>>>>>>> On 01-02-2020 02:23, RAJESWARAN THILLAIGOVINDAN wrote:
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> I am going through the bmcweb code for implementing Redfish
>>>>>>>>>>> EventService based on the design document
>>>>>>>>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749.
>>>>>>>>>>> This design is hooked to the journal based Redfish Event
>>>>>>>>>>> Logging. For life cycle events(ResourceAdded,
>>>>>>>>>>> ResourceRemoved, ResourceUpdated), using D-Bus match,
>>>>>>>>>>> bmcweb can create an event log. This requires a JSON
>>>>>>>>>>> dictionary, comprising an array of Redfish Resource Name and
>>>>>>>>>>> the D-Bus path. This approach works only in case of one to
>>>>>>>>>>> one mapping of Redfish Resource Name and the D-Bus path. For
>>>>>>>>>>> propertiesChanged events, if the Redfish Resource property
>>>>>>>>>>> is not on the same D-Bus path or the Redfish Resource
>>>>>>>>>>> property name is different from the D-Bus property name,
>>>>>>>>>>> then an additional JSON dictionary to maintain this
>>>>>>>>>>> information is required. With D-Bus match alone in the
>>>>>>>>>>> bmcweb, Redfish EventService can't be fully supported. For
>>>>>>>>>>> the Message Registers and the Resource Types that are
>>>>>>>>>>> supported, the relevant OpenBMC application must create an
>>>>>>>>>>> event log in the journal using either the
>>>>>>>>>>> phosphor::logging::entry or sd_journal_send() command.
>>>>>>>>>>>
>>>>>>>>>>> After realizing that with D-Bus match in the bmcweb alone
>>>>>>>>>>> can't help to fully implement EventService, I prefer to
>>>>>>>>>>> avoid using D-Bus match in bmcweb. Instead, I prefer to
>>>>>>>>>>> modify the OpenBMC application that generated the event to
>>>>>>>>>>> create an event log in the journal. Do you see any advantage
>>>>>>>>>>> of using combination of D-Bus match in the bmcweb wherever
>>>>>>>>>>> it is possible and changes to OpenBMC application in other
>>>>>>>>>>> cases to create an event log ?
>>>>>>>>>>>
>>>>>>>>>>> Your views are highly appreciated.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Rajes
>>>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> Ratan
>>>>>>>>
>>>>>>>>
>> Ratan
[-- Attachment #2: Type: text/html, Size: 24283 bytes --]
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-01-31 20:53 Redfish EventService Implementation RAJESWARAN THILLAIGOVINDAN
2020-02-09 20:22 ` RAJESWARAN THILLAIGOVINDAN
@ 2020-06-08 21:08 ` Brad Bishop
2020-06-09 0:58 ` James Feist
1 sibling, 1 reply; 44+ messages in thread
From: Brad Bishop @ 2020-06-08 21:08 UTC (permalink / raw)
To: RAJESWARAN THILLAIGOVINDAN, openbmc, apparao.puli
On Sat, 2020-02-01 at 02:23 +0530, RAJESWARAN THILLAIGOVINDAN wrote:
> Hi,
>
> I am going through the bmcweb code for implementing Redfish
> EventService based on the design document
> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design
> is hooked to the journal based Redfish Event Logging.
Would anyone else be willing to opine on whether or not they think
journal based event schemes are what we want going forward for OpenBMC?
My feeling is that they are not - as an alternative IPC mechanism don't
we end up re-implementing things that DBus already does? Doesn't it
require us to raise the same event twice everywhere (Once with DBus, and
once in the journal)? What does journal based eventing do that DBus
signals don't do?
Please poke holes.
thx - brad
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-08 21:08 ` Redfish EventService Implementation Brad Bishop
@ 2020-06-09 0:58 ` James Feist
2020-06-15 12:50 ` Ratan Gupta
0 siblings, 1 reply; 44+ messages in thread
From: James Feist @ 2020-06-09 0:58 UTC (permalink / raw)
To: Brad Bishop, RAJESWARAN THILLAIGOVINDAN, openbmc, apparao.puli
Cc: Bills, Jason M
On 6/8/2020 2:08 PM, Brad Bishop wrote:
> On Sat, 2020-02-01 at 02:23 +0530, RAJESWARAN THILLAIGOVINDAN wrote:
>> Hi,
>>
>> I am going through the bmcweb code for implementing Redfish
>> EventService based on the design document
>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design
>> is hooked to the journal based Redfish Event Logging.
>
> Would anyone else be willing to opine on whether or not they think
> journal based event schemes are what we want going forward for OpenBMC?
>
> My feeling is that they are not - as an alternative IPC mechanism don't
> we end up re-implementing things that DBus already does? Doesn't it
> require us to raise the same event twice everywhere (Once with DBus, and
> once in the journal)? What does journal based eventing do that DBus
> signals don't do?
We don't host log events on DBus at all, so there is no duplicate. The
journal gives built in persistence and rotating of logs for large number
of events. I know when this came up the last time one of the big issues
was supporting thousands of logs wouldn't work well on DBus.
>
> Please poke holes.
>
> thx - brad
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-09 0:58 ` James Feist
@ 2020-06-15 12:50 ` Ratan Gupta
2020-06-15 21:42 ` James Feist
0 siblings, 1 reply; 44+ messages in thread
From: Ratan Gupta @ 2020-06-15 12:50 UTC (permalink / raw)
To: James Feist, Brad Bishop, Puli, Apparao, openbmc, apparao.puli
Cc: Bills, Jason M
Hi James/Apparao/Brad,
I am inclining towards having a separate application for Redfish
Logs(like: phosphor-sel-logger),
This application does the following.
1) Have the mapping info from Redfish resources to Dbus Resources
* This is needed as webserver have to provide the event filtering
through Resource Type
* eg : Redfish Client may ask as the client is interested in
"Account" Resource type
i.e all the user account related updates should be given to
redfish client.
which suggest that there should be mapping from Redfish
Resource to Dbus Resource
2) Have the reverse mapping from Dbus Resources to Redfish Resources
* This is needed to send the Redfish event if there is any changes
in the
corresponding D-bus resources. eg BMC state change/network
change etc.
3) This application monitors the D-bus event and Log the event in the
journal like below
eg:
sd_journal_send("MESSAGE=%s", "Account Modified",
"PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s",
"REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>",
"REDFISH_RESOURCE_TYPE=ComputerSystem"
"REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem",
"REDFISH_MESSAGE_ARGS=%s", "Off", NULL);
4) rsyslogd will put all these logs from journal into "/var/log/redfish"
file.
5) bmcweb/webserver would inotify this file location and on notification
it will send redfish event
6) Event filtering would be done at the bmcweb/webserver side.
We already have the infrastructure for seq no 4 and seq no 5 and we
wanted to leverage this infrastructure.
Please let me know if there is any concern with this approach.
Ratan
On 6/9/20 6:28 AM, James Feist wrote:
> On 6/8/2020 2:08 PM, Brad Bishop wrote:
>> On Sat, 2020-02-01 at 02:23 +0530, RAJESWARAN THILLAIGOVINDAN wrote:
>>> Hi,
>>>
>>> I am going through the bmcweb code for implementing Redfish
>>> EventService based on the design document
>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design
>>> is hooked to the journal based Redfish Event Logging.
>>
>> Would anyone else be willing to opine on whether or not they think
>> journal based event schemes are what we want going forward for OpenBMC?
>>
>> My feeling is that they are not - as an alternative IPC mechanism don't
>> we end up re-implementing things that DBus already does? Doesn't it
>> require us to raise the same event twice everywhere (Once with DBus, and
>> once in the journal)? What does journal based eventing do that DBus
>> signals don't do?
>
> We don't host log events on DBus at all, so there is no duplicate. The
> journal gives built in persistence and rotating of logs for large
> number of events. I know when this came up the last time one of the
> big issues was supporting thousands of logs wouldn't work well on DBus.
>
>
>>
>> Please poke holes.
>>
>> thx - brad
>>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-15 12:50 ` Ratan Gupta
@ 2020-06-15 21:42 ` James Feist
2020-06-16 15:24 ` Patrick Williams
0 siblings, 1 reply; 44+ messages in thread
From: James Feist @ 2020-06-15 21:42 UTC (permalink / raw)
To: Ratan Gupta, Brad Bishop, Puli, Apparao, openbmc; +Cc: Bills, Jason M
On 6/15/2020 5:50 AM, Ratan Gupta wrote:
> Hi James/Apparao/Brad,
>
> I am inclining towards having a separate application for Redfish
> Logs(like: phosphor-sel-logger),
> This application does the following.
>
> 1) Have the mapping info from Redfish resources to Dbus Resources
> * This is needed as webserver have to provide the event filtering
> through Resource Type
> * eg : Redfish Client may ask as the client is interested in
> "Account" Resource type
> i.e all the user account related updates should be given to
> redfish client.
> which suggest that there should be mapping from Redfish
> Resource to Dbus Resource
>
> 2) Have the reverse mapping from Dbus Resources to Redfish Resources
> * This is needed to send the Redfish event if there is any changes
> in the
> corresponding D-bus resources. eg BMC state change/network
> change etc.
>
> 3) This application monitors the D-bus event and Log the event in the
> journal like below
Why do we need a listener on d-bus if the application creating the event
already knows all the information? Why can't it log straight to the journal?
> eg:
> sd_journal_send("MESSAGE=%s", "Account Modified",
> "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s",
> "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>",
> "REDFISH_RESOURCE_TYPE=ComputerSystem"
> "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem",
> "REDFISH_MESSAGE_ARGS=%s", "Off", NULL);
If we're going to go down the path of re-implementing logging, I think
the goal should be to stop logging things in the Journal that are
Redfish specific, and instead log them in some generic format that
phosphor logging controls the map. Right now we are bifurcated because
the dbus-event-logs, SEL, PEL, and Redfish are all using different
methods of logging, that play to their own set of rules. Most repos use
phosphor-logging, so instead of creating yet another daemon, if we added
support to create a 'System Level' or 'External User' log that has
predefined dictionaries of required and optional keys, something like
phosphor-dbus-interfaces, we might be able to drop some of these
transport specific logs, and have the transports based on the same
database (the journal). Then each transport could filter these based on
journal entry type, and transform them into the correct log for that
transport. I think adding more Redfish specifics into the logs hinders
those who do not want Redfish in their systems.
>
> 4) rsyslogd will put all these logs from journal into "/var/log/redfish"
> file.
>
> 5) bmcweb/webserver would inotify this file location and on notification
> it will send redfish event
>
> 6) Event filtering would be done at the bmcweb/webserver side.
>
I think this will probably be needed if we want one source of truth for
logs.
>
> We already have the infrastructure for seq no 4 and seq no 5 and we
> wanted to leverage this infrastructure.
>
> Please let me know if there is any concern with this approach.
>
> Ratan
>
> On 6/9/20 6:28 AM, James Feist wrote:
>> On 6/8/2020 2:08 PM, Brad Bishop wrote:
>>> On Sat, 2020-02-01 at 02:23 +0530, RAJESWARAN THILLAIGOVINDAN wrote:
>>>> Hi,
>>>>
>>>> I am going through the bmcweb code for implementing Redfish
>>>> EventService based on the design document
>>>> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749. This design
>>>> is hooked to the journal based Redfish Event Logging.
>>>
>>> Would anyone else be willing to opine on whether or not they think
>>> journal based event schemes are what we want going forward for OpenBMC?
>>>
>>> My feeling is that they are not - as an alternative IPC mechanism don't
>>> we end up re-implementing things that DBus already does? Doesn't it
>>> require us to raise the same event twice everywhere (Once with DBus, and
>>> once in the journal)? What does journal based eventing do that DBus
>>> signals don't do?
>>
>> We don't host log events on DBus at all, so there is no duplicate. The
>> journal gives built in persistence and rotating of logs for large
>> number of events. I know when this came up the last time one of the
>> big issues was supporting thousands of logs wouldn't work well on DBus.
>>
>>
>>>
>>> Please poke holes.
>>>
>>> thx - brad
>>>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-15 21:42 ` James Feist
@ 2020-06-16 15:24 ` Patrick Williams
2020-06-16 16:11 ` James Feist
0 siblings, 1 reply; 44+ messages in thread
From: Patrick Williams @ 2020-06-16 15:24 UTC (permalink / raw)
To: James Feist
Cc: Ratan Gupta, Brad Bishop, Puli, Apparao, openbmc, Bills, Jason M
[-- Attachment #1: Type: text/plain, Size: 2365 bytes --]
On Mon, Jun 15, 2020 at 02:42:11PM -0700, James Feist wrote:
> On 6/15/2020 5:50 AM, Ratan Gupta wrote:
> > eg:
> > sd_journal_send("MESSAGE=%s", "Account Modified",
> > "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s",
> > "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>",
> > "REDFISH_RESOURCE_TYPE=ComputerSystem"
> > "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem",
> > "REDFISH_MESSAGE_ARGS=%s", "Off", NULL);
>
> If we're going to go down the path of re-implementing logging, I think
> the goal should be to stop logging things in the Journal that are
> Redfish specific, and instead log them in some generic format that
> phosphor logging controls the map. Right now we are bifurcated because
> the dbus-event-logs, SEL, PEL, and Redfish are all using different
> methods of logging, that play to their own set of rules.
Absolutely agree with you here. There is zero reason that applications
should start logging specially formed messages with REDFISH_* meta-data.
We shouldn't have any applications explicitly know about Redfish except
the Redfish providers themselves. This is no different from IPMI, PLDM,
or any other external interface.
The kind of information presented here as being interesting to expose
via Redfish is equally as interesting to me to be able to add to one of
our 'FFDC dumps', which could be used for security / forensic work.
> Most repos use
> phosphor-logging, so instead of creating yet another daemon, if we added
> support to create a 'System Level' or 'External User' log that has
> predefined dictionaries of required and optional keys, something like
> phosphor-dbus-interfaces, we might be able to drop some of these
> transport specific logs, and have the transports based on the same
> database (the journal). Then each transport could filter these based on
> journal entry type, and transform them into the correct log for that
> transport. I think adding more Redfish specifics into the logs hinders
> those who do not want Redfish in their systems.
Can't we do this already today by defining a simple errors/metadata file
in phosphor-dbus-interfaces and calling 'logging::report<...>' on it?
This will create a record on dbus in phosphor-logging.
--
Patrick Williams
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-16 15:24 ` Patrick Williams
@ 2020-06-16 16:11 ` James Feist
2020-06-17 12:08 ` Ratan Gupta
0 siblings, 1 reply; 44+ messages in thread
From: James Feist @ 2020-06-16 16:11 UTC (permalink / raw)
To: Patrick Williams
Cc: Bills, Jason M, openbmc, Brad Bishop, Puli, Apparao, Ratan Gupta
On 6/16/2020 8:24 AM, Patrick Williams wrote:
> On Mon, Jun 15, 2020 at 02:42:11PM -0700, James Feist wrote:
>> On 6/15/2020 5:50 AM, Ratan Gupta wrote:
>>> eg:
>>> sd_journal_send("MESSAGE=%s", "Account Modified",
>>> "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s",
>>> "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>",
>>> "REDFISH_RESOURCE_TYPE=ComputerSystem"
>>> "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem",
>>> "REDFISH_MESSAGE_ARGS=%s", "Off", NULL);
>>
>> If we're going to go down the path of re-implementing logging, I think
>> the goal should be to stop logging things in the Journal that are
>> Redfish specific, and instead log them in some generic format that
>> phosphor logging controls the map. Right now we are bifurcated because
>> the dbus-event-logs, SEL, PEL, and Redfish are all using different
>> methods of logging, that play to their own set of rules.
>
> Absolutely agree with you here. There is zero reason that applications
> should start logging specially formed messages with REDFISH_* meta-data.
> We shouldn't have any applications explicitly know about Redfish except
> the Redfish providers themselves. This is no different from IPMI, PLDM,
> or any other external interface.
>
> The kind of information presented here as being interesting to expose
> via Redfish is equally as interesting to me to be able to add to one of
> our 'FFDC dumps', which could be used for security / forensic work.
>
>> Most repos use
>> phosphor-logging, so instead of creating yet another daemon, if we added
>> support to create a 'System Level' or 'External User' log that has
>> predefined dictionaries of required and optional keys, something like
>> phosphor-dbus-interfaces, we might be able to drop some of these
>> transport specific logs, and have the transports based on the same
>> database (the journal). Then each transport could filter these based on
>> journal entry type, and transform them into the correct log for that
>> transport. I think adding more Redfish specifics into the logs hinders
>> those who do not want Redfish in their systems.
>
> Can't we do this already today by defining a simple errors/metadata file
> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it?
> This will create a record on dbus in phosphor-logging.
>
I think the original concern was with supporting on the order of 10,000
log entries, having this on d-bus seemed impractical. Also the free log
rotation the journal provides is useful. Now modifying the
logging::report<...> to conditionally log to the journal seems realistic.
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-16 16:11 ` James Feist
@ 2020-06-17 12:08 ` Ratan Gupta
2020-06-17 17:16 ` James Feist
` (2 more replies)
0 siblings, 3 replies; 44+ messages in thread
From: Ratan Gupta @ 2020-06-17 12:08 UTC (permalink / raw)
To: James Feist, Patrick Williams
Cc: Bills, Jason M, openbmc, Brad Bishop, Puli, Apparao
Hi James,Pattrick.
Thanks for giving the feedback
On 6/16/20 9:41 PM, James Feist wrote:
> On 6/16/2020 8:24 AM, Patrick Williams wrote:
>> On Mon, Jun 15, 2020 at 02:42:11PM -0700, James Feist wrote:
>>> On 6/15/2020 5:50 AM, Ratan Gupta wrote:
>>>> eg:
>>>> sd_journal_send("MESSAGE=%s", "Account Modified",
>>>> "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s",
>>>> "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>",
>>>> "REDFISH_RESOURCE_TYPE=ComputerSystem"
>>>> "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem",
>>>> "REDFISH_MESSAGE_ARGS=%s", "Off", NULL);
>>>
>>> If we're going to go down the path of re-implementing logging, I think
>>> the goal should be to stop logging things in the Journal that are
>>> Redfish specific, and instead log them in some generic format that
>>> phosphor logging controls the map. Right now we are bifurcated because
>>> the dbus-event-logs, SEL, PEL, and Redfish are all using different
>>> methods of logging, that play to their own set of rules.
>>
>> Absolutely agree with you here. There is zero reason that applications
>> should start logging specially formed messages with REDFISH_* meta-data.
>> We shouldn't have any applications explicitly know about Redfish except
>> the Redfish providers themselves. This is no different from IPMI, PLDM,
>> or any other external interface.
>>
>> The kind of information presented here as being interesting to expose
>> via Redfish is equally as interesting to me to be able to add to one of
>> our 'FFDC dumps', which could be used for security / forensic work.
>>
>>> Most repos use
>>> phosphor-logging, so instead of creating yet another daemon, if we
>>> added
>>> support to create a 'System Level' or 'External User' log that has
>>> predefined dictionaries of required and optional keys, something like
>>> phosphor-dbus-interfaces, we might be able to drop some of these
>>> transport specific logs, and have the transports based on the same
>>> database (the journal). Then each transport could filter these based on
>>> journal entry type, and transform them into the correct log for that
>>> transport. I think adding more Redfish specifics into the logs hinders
>>> those who do not want Redfish in their systems.
>>
>> Can't we do this already today by defining a simple errors/metadata file
>> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it?
>> This will create a record on dbus in phosphor-logging.
>>
> I think the original concern was with supporting on the order of
> 10,000 log entries, having this on d-bus seemed impractical. Also the
> free log rotation the journal provides is useful. Now modifying the
> logging::report<...> to conditionally log to the journal seems realistic.
My intention was not to re-implement the logging, my intention was to
extend/use the existing design which we already have it below.
https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md
I was trying not to bring the Redfish specific stuff in each individual
repo, instead each transport can listen for
Dbus events and write to the journal which goes to their app specific file.
Your point is valid that we would be polluting the journal if each and
every transport start writing as per their needs.
***** As per the Redfish one of the requirement is we need the log for
most of the Dbus Property update/interface added as they
are mapped to some Redfish Resource and the bmcweb has to send the
Resource updated/modified signal to the
Redfish client. ******
As we are in agreement that we want to use the journal for persistence
and log rotate feature.
We have two options:
1) Each transport interface listens for the Dbus signals and write
it to their app specific file.
2) Each openbmc repo must use log::report for each D-bus property
update/ interface added.
To make this happen, we need the following
* common message format which can be used by the all other
application (SEL,Redfish) etc.
* Transport application will read this common message from the
journal and use it as per their needs.
Option no 2: It is ideal solution but that requires good amount of
work as
either each and every D-bus repo should write to the
journal for any property update/interface added.
or
Should SDbusplus do this as each and every application
uses the sdbusplus framework to manage the dbus objects
Please let me know your thoughts.
Ratan
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-17 12:08 ` Ratan Gupta
@ 2020-06-17 17:16 ` James Feist
2020-06-17 17:19 ` Bills, Jason M
2020-06-17 20:45 ` Patrick Williams
2 siblings, 0 replies; 44+ messages in thread
From: James Feist @ 2020-06-17 17:16 UTC (permalink / raw)
To: Ratan Gupta, Patrick Williams
Cc: Puli, Apparao, Bills, Jason M, Brad Bishop, openbmc
> My intention was not to re-implement the logging, my intention was to
> extend/use the existing design which we already have it below.
>
> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md
>
>
> I was trying not to bring the Redfish specific stuff in each individual
> repo, instead each transport can listen for
> Dbus events and write to the journal which goes to their app specific file.
>
> Your point is valid that we would be polluting the journal if each and
> every transport start writing as per their needs.
>
> ***** As per the Redfish one of the requirement is we need the log for
> most of the Dbus Property update/interface added as they
> are mapped to some Redfish Resource and the bmcweb has to send the
> Resource updated/modified signal to the
> Redfish client. ******
>
> As we are in agreement that we want to use the journal for persistence
> and log rotate feature.
>
> We have two options:
> 1) Each transport interface listens for the Dbus signals and write
> it to their app specific file.
> 2) Each openbmc repo must use log::report for each D-bus property
> update/ interface added.
I'm assuming you meant each event added right? I don't see any reason
that the logs have anything to do with d-bus, except maybe if it seems
worth it for phosphor-logging to catch all the reports and log them to
the journal. Although it would probably be more efficient (and easier
for phased adoption) to just make the log::report call write to the
journal as well.
> To make this happen, we need the following
> * common message format which can be used by the all other
> application (SEL,Redfish) etc.
Agree.
> * Transport application will read this common message from the
> journal and use it as per their needs.
>
> Option no 2: It is ideal solution but that requires good amount of
> work as
> either each and every D-bus repo should write to the
> journal for any property update/interface added.
Why every property/interface added? This would make the logs much to
large to be useful. And many times the information needed is specific to
the log type. For instance a event for inventory discovered might need
the type of inventory, the serial number, the model, etc. To get all
that information would be multiple interfaces added events combined
together. It would make more sense in this aspect for the inventory
daemon to take the parts out of say the Asset, and Chassis interfaces
along with the path identifier, and create an entry in the log that is
use-able. It would be much more difficult and error-prone for the
Redfish or SEL daemon to try to piece these multiple events together to
form something useful.
> or
> Should SDbusplus do this as each and every application
> uses the sdbusplus framework to manage the dbus objects
>
> Please let me know your thoughts.
>
> Ratan
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-17 12:08 ` Ratan Gupta
2020-06-17 17:16 ` James Feist
@ 2020-06-17 17:19 ` Bills, Jason M
2020-06-17 18:30 ` Andrew Geissler
2020-06-17 20:45 ` Patrick Williams
2 siblings, 1 reply; 44+ messages in thread
From: Bills, Jason M @ 2020-06-17 17:19 UTC (permalink / raw)
To: openbmc
On 6/17/2020 5:08 AM, Ratan Gupta wrote:
> Hi James,Pattrick.
>
> Thanks for giving the feedback
>
> On 6/16/20 9:41 PM, James Feist wrote:
>> On 6/16/2020 8:24 AM, Patrick Williams wrote:
>>> On Mon, Jun 15, 2020 at 02:42:11PM -0700, James Feist wrote:
>>>> On 6/15/2020 5:50 AM, Ratan Gupta wrote:
>>>>> eg:
>>>>> sd_journal_send("MESSAGE=%s", "Account Modified",
>>>>> "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s",
>>>>> "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>",
>>>>> "REDFISH_RESOURCE_TYPE=ComputerSystem"
>>>>> "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem",
>>>>> "REDFISH_MESSAGE_ARGS=%s", "Off", NULL);
>>>>
>>>> If we're going to go down the path of re-implementing logging, I think
>>>> the goal should be to stop logging things in the Journal that are
>>>> Redfish specific, and instead log them in some generic format that
>>>> phosphor logging controls the map. Right now we are bifurcated because
>>>> the dbus-event-logs, SEL, PEL, and Redfish are all using different
>>>> methods of logging, that play to their own set of rules.
>>>
>>> Absolutely agree with you here. There is zero reason that applications
>>> should start logging specially formed messages with REDFISH_* meta-data.
>>> We shouldn't have any applications explicitly know about Redfish except
>>> the Redfish providers themselves. This is no different from IPMI, PLDM,
>>> or any other external interface.
>>>
>>> The kind of information presented here as being interesting to expose
>>> via Redfish is equally as interesting to me to be able to add to one of
>>> our 'FFDC dumps', which could be used for security / forensic work.
>>>
>>>> Most repos use
>>>> phosphor-logging, so instead of creating yet another daemon, if we
>>>> added
>>>> support to create a 'System Level' or 'External User' log that has
>>>> predefined dictionaries of required and optional keys, something like
>>>> phosphor-dbus-interfaces, we might be able to drop some of these
>>>> transport specific logs, and have the transports based on the same
>>>> database (the journal). Then each transport could filter these based on
>>>> journal entry type, and transform them into the correct log for that
>>>> transport. I think adding more Redfish specifics into the logs hinders
>>>> those who do not want Redfish in their systems.
>>>
>>> Can't we do this already today by defining a simple errors/metadata file
>>> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it?
>>> This will create a record on dbus in phosphor-logging.
>>>
>> I think the original concern was with supporting on the order of
>> 10,000 log entries, having this on d-bus seemed impractical. Also the
>> free log rotation the journal provides is useful. Now modifying the
>> logging::report<...> to conditionally log to the journal seems realistic.
> My intention was not to re-implement the logging, my intention was to
> extend/use the existing design which we already have it below.
>
> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md
>
>
> I was trying not to bring the Redfish specific stuff in each individual
> repo, instead each transport can listen for
> Dbus events and write to the journal which goes to their app specific file.
>
> Your point is valid that we would be polluting the journal if each and
> every transport start writing as per their needs.
>
> ***** As per the Redfish one of the requirement is we need the log for
> most of the Dbus Property update/interface added as they
> are mapped to some Redfish Resource and the bmcweb has to send the
> Resource updated/modified signal to the
> Redfish client. ******
Why do we want to log on D-Bus property updates? This seems like it
will be too noisy for the EventLog.
>
> As we are in agreement that we want to use the journal for persistence
> and log rotate feature.
>
> We have two options:
> 1) Each transport interface listens for the Dbus signals and write
> it to their app specific file.
> 2) Each openbmc repo must use log::report for each D-bus property
> update/ interface added.
> To make this happen, we need the following
> * common message format which can be used by the all other
> application (SEL,Redfish) etc.
> * Transport application will read this common message from the
> journal and use it as per their needs.
>
> Option no 2: It is ideal solution but that requires good amount of
> work as
> either each and every D-bus repo should write to the
> journal for any property update/interface added.
> or
> Should SDbusplus do this as each and every application
> uses the sdbusplus framework to manage the dbus objects
>
> Please let me know your thoughts.
>
> Ratan
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-17 17:19 ` Bills, Jason M
@ 2020-06-17 18:30 ` Andrew Geissler
0 siblings, 0 replies; 44+ messages in thread
From: Andrew Geissler @ 2020-06-17 18:30 UTC (permalink / raw)
To: Bills, Jason M; +Cc: openbmc
> On Jun 17, 2020, at 12:19 PM, Bills, Jason M <jason.m.bills@linux.intel.com> wrote:
>
>
>
> On 6/17/2020 5:08 AM, Ratan Gupta wrote:
>> Hi James,Pattrick.
>> Thanks for giving the feedback
>> On 6/16/20 9:41 PM, James Feist wrote:
>>> On 6/16/2020 8:24 AM, Patrick Williams wrote:
>>>> On Mon, Jun 15, 2020 at 02:42:11PM -0700, James Feist wrote:
>>>>> On 6/15/2020 5:50 AM, Ratan Gupta wrote:
>>>>>> eg:
>>>>>> sd_journal_send("MESSAGE=%s", "Account Modified",
>>>>>> "PRIORITY=%i", LOG_INFO, "REDFISH_MESSAGE_ID=%s",
>>>>>> "REDFISH_RESOURCE_PATH=/redfish/v1/AccountService/accounts/<id>",
>>>>>> "REDFISH_RESOURCE_TYPE=ComputerSystem"
>>>>>> "REDFISH_REGISTRY_PREFIX=Task/Base/Resource/Oem",
>>>>>> "REDFISH_MESSAGE_ARGS=%s", "Off", NULL);
>>>>>
>>>>> If we're going to go down the path of re-implementing logging, I think
>>>>> the goal should be to stop logging things in the Journal that are
>>>>> Redfish specific, and instead log them in some generic format that
>>>>> phosphor logging controls the map. Right now we are bifurcated because
>>>>> the dbus-event-logs, SEL, PEL, and Redfish are all using different
>>>>> methods of logging, that play to their own set of rules.
>>>>
>>>> Absolutely agree with you here. There is zero reason that applications
>>>> should start logging specially formed messages with REDFISH_* meta-data.
>>>> We shouldn't have any applications explicitly know about Redfish except
>>>> the Redfish providers themselves. This is no different from IPMI, PLDM,
>>>> or any other external interface.
>>>>
>>>> The kind of information presented here as being interesting to expose
>>>> via Redfish is equally as interesting to me to be able to add to one of
>>>> our 'FFDC dumps', which could be used for security / forensic work.
>>>>
>>>>> Most repos use
>>>>> phosphor-logging, so instead of creating yet another daemon, if we added
>>>>> support to create a 'System Level' or 'External User' log that has
>>>>> predefined dictionaries of required and optional keys, something like
>>>>> phosphor-dbus-interfaces, we might be able to drop some of these
>>>>> transport specific logs, and have the transports based on the same
>>>>> database (the journal). Then each transport could filter these based on
>>>>> journal entry type, and transform them into the correct log for that
>>>>> transport. I think adding more Redfish specifics into the logs hinders
>>>>> those who do not want Redfish in their systems.
>>>>
>>>> Can't we do this already today by defining a simple errors/metadata file
>>>> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it?
>>>> This will create a record on dbus in phosphor-logging.
>>>>
>>> I think the original concern was with supporting on the order of 10,000 log entries, having this on d-bus seemed impractical. Also the free log rotation the journal provides is useful. Now modifying the logging::report<...> to conditionally log to the journal seems realistic.
>> My intention was not to re-implement the logging, my intention was to extend/use the existing design which we already have it below.
>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md I was trying not to bring the Redfish specific stuff in each individual repo, instead each transport can listen for
>> Dbus events and write to the journal which goes to their app specific file.
>> Your point is valid that we would be polluting the journal if each and every transport start writing as per their needs.
>> ***** As per the Redfish one of the requirement is we need the log for most of the Dbus Property update/interface added as they
>> are mapped to some Redfish Resource and the bmcweb has to send the Resource updated/modified signal to the
>> Redfish client. ******
> Why do we want to log on D-Bus property updates? This seems like it will be too noisy for the EventLog.
I don’t think it would be all, just the one’s the different transport layers need.
So to get my head straight on this, there is a commit in state-manager to log
the redfish events to the journal:
https://gerrit.openbmc-project.xyz/c/openbmc/phosphor-state-manager/+/33494
With what Ratan proposed above, we have two options?
1) A separate application that listens for the chassis properties to change and logs those
events to the journal in the format the transport layer wants
2) Code within phosphor-state-manager that logs a generic event to the
journal which then can be post processed by the different transport
layer code.
>
>> As we are in agreement that we want to use the journal for persistence and log rotate feature.
>> We have two options:
>> 1) Each transport interface listens for the Dbus signals and write it to their app specific file.
>> 2) Each openbmc repo must use log::report for each D-bus property update/ interface added.
>> To make this happen, we need the following
>> * common message format which can be used by the all other application (SEL,Redfish) etc.
>> * Transport application will read this common message from the journal and use it as per their needs.
>> Option no 2: It is ideal solution but that requires good amount of work as
>> either each and every D-bus repo should write to the journal for any property update/interface added.
>> or
>> Should SDbusplus do this as each and every application uses the sdbusplus framework to manage the dbus objects
>> Please let me know your thoughts.
>> Ratan
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-17 12:08 ` Ratan Gupta
2020-06-17 17:16 ` James Feist
2020-06-17 17:19 ` Bills, Jason M
@ 2020-06-17 20:45 ` Patrick Williams
2020-06-19 13:26 ` Ratan Gupta
2 siblings, 1 reply; 44+ messages in thread
From: Patrick Williams @ 2020-06-17 20:45 UTC (permalink / raw)
To: Ratan Gupta
Cc: James Feist, Bills, Jason M, openbmc, Brad Bishop, Puli, Apparao
[-- Attachment #1: Type: text/plain, Size: 3658 bytes --]
On Wed, Jun 17, 2020 at 05:38:47PM +0530, Ratan Gupta wrote:
> Hi James,Pattrick.
>
> >> Can't we do this already today by defining a simple errors/metadata file
> >> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it?
> >> This will create a record on dbus in phosphor-logging.
> >>
> > I think the original concern was with supporting on the order of
> > 10,000 log entries, having this on d-bus seemed impractical. Also the
> > free log rotation the journal provides is useful. Now modifying the
> > logging::report<...> to conditionally log to the journal seems realistic.
> My intention was not to re-implement the logging, my intention was to
> extend/use the existing design which we already have it below.
>
> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md
>
> I was trying not to bring the Redfish specific stuff in each individual
> repo, instead each transport can listen for
> Dbus events and write to the journal which goes to their app specific file.
Good. This wasn't clear from the earlier email. Thanks.
> As we are in agreement that we want to use the journal for persistence
> and log rotate feature.
I'm not convinced there is agreement on this. There has been
disagreement about even using the journal for phosphor-logging use since
the beginning and I suspect there would be less agreement on another
application using it as its own IPC mechanism.
Just because a hammer can be used to insert a nail into a board doesn't
mean you use it insert any pointy thing into a flat thing. [ Just
because the journal provides log rotation and persistance doesn't mean
you should use it for every feature needing log rotation and
persistance. ]
> ***** As per the Redfish one of the requirement is we need the log for
> most of the Dbus Property update/interface added as they
> are mapped to some Redfish Resource and the bmcweb has to send the
> Resource updated/modified signal to the
> Redfish client. ******
I don't know Redfish well, so bear with me if there is something obvious
I'm missing. But, the first part of this "requirement" doesn't seem to
follow from the second part of the "requirement" to me.
Sending a signal of a property changing to the Redfish clients is
straight-forwawrd; Redfish should subscribe to all the appropriate
dbus-events. I don't understand how this implies any sort of logging.
Where does the logging part of this requirement come from?
> We have two options:
> 1) Each transport interface listens for the Dbus signals and write
> it to their app specific file.
> 2) Each openbmc repo must use log::report for each D-bus property
> update/ interface added.
#2 is absolutely unworkable on the surface to me. log::report is to
create a error entry (xyz.openbmc_project.Logging.Entry), which creates
a dbus-object, which would cause log::report to be called, which creates
a dbus-object, which ...
Even if what you meant was something like logging::log<info>, this seems
pretty heavy. I'm not sure this is something that can be inserted into
sdbusplus, especially for the ASIO-based object servers, because in many
cases applications register their own callback. For the sdbus++
generated server bindings we could squeeze it in. But, what you're
proposing here is essentially a "journal-as-dbus-monitor". We'd
certainly need to make some measurements on how many kB/s worth of
journal entries this would create because I suspect we could end up
burning out a NAND flash with as many writes as that would induce.
--
Patrick Williams
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-17 20:45 ` Patrick Williams
@ 2020-06-19 13:26 ` Ratan Gupta
2020-06-19 22:19 ` Bills, Jason M
2020-06-23 7:27 ` Ratan Gupta
0 siblings, 2 replies; 44+ messages in thread
From: Ratan Gupta @ 2020-06-19 13:26 UTC (permalink / raw)
To: Patrick Williams
Cc: Puli, Apparao, Bills, Jason M, Brad Bishop, James Feist, openbmc
[-- Attachment #1: Type: text/plain, Size: 5106 bytes --]
On 6/18/20 2:15 AM, Patrick Williams wrote:
> On Wed, Jun 17, 2020 at 05:38:47PM +0530, Ratan Gupta wrote:
>> Hi James,Pattrick.
>>
>>>> Can't we do this already today by defining a simple errors/metadata file
>>>> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it?
>>>> This will create a record on dbus in phosphor-logging.
>>>>
>>> I think the original concern was with supporting on the order of
>>> 10,000 log entries, having this on d-bus seemed impractical. Also the
>>> free log rotation the journal provides is useful. Now modifying the
>>> logging::report<...> to conditionally log to the journal seems realistic.
>> My intention was not to re-implement the logging, my intention was to
>> extend/use the existing design which we already have it below.
>>
>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md
>>
>> I was trying not to bring the Redfish specific stuff in each individual
>> repo, instead each transport can listen for
>> Dbus events and write to the journal which goes to their app specific file.
> Good. This wasn't clear from the earlier email. Thanks.
>
>
>> As we are in agreement that we want to use the journal for persistence
>> and log rotate feature.
> I'm not convinced there is agreement on this. There has been
> disagreement about even using the journal for phosphor-logging use since
> the beginning and I suspect there would be less agreement on another
> application using it as its own IPC mechanism.
>
> Just because a hammer can be used to insert a nail into a board doesn't
> mean you use it insert any pointy thing into a flat thing. [ Just
> because the journal provides log rotation and persistance doesn't mean
> you should use it for every feature needing log rotation and
> persistance. ]
>
>
>> ***** As per the Redfish one of the requirement is we need the log for
>> most of the Dbus Property update/interface added as they
>> are mapped to some Redfish Resource and the bmcweb has to send the
>> Resource updated/modified signal to the
>> Redfish client. ******
Jaosn: You asked the following query in other thread /*"Why do we want
to log on D-Bus property updates? This seems like it will be too noisy
for the EventLog*"/
Eg: Client is interested for an event when ever there is any user
add/delete or network configuration change or there is a log entry
resource gets created,To handle this request the flow would be
Redfish Client subscribe for "ResourceType" eg:
"EthernetService,AccountService,LogService" with subordinate resources
property as truewhich means the Client is looking for updates on the
subscribed resources and the subordinates resource, These redfish
resources(EthernetInterface, IP address, ManagerAccount, AccountService)
would be mapped to some D-bus Resources, hence some application/bmcweb
would monitor the Dbus signals on the interested Dbus objects and send
the Redfish event to the subscribed client.
Apparo: Please correct me if I am missing something.
> I don't know Redfish well, so bear with me if there is something obvious
> I'm missing. But, the first part of this "requirement" doesn't seem to
> follow from the second part of the "requirement" to me.
>
> Sending a signal of a property changing to the Redfish clients is
> straight-forwawrd; Redfish should subscribe to all the appropriate
> dbus-events. I don't understand how this implies any sort of logging.
> Where does the logging part of this requirement come from?
https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749/16/designs/redfish-eventservice.md#474
While I am reading the redfish
spec(https://www.dmtf.org/sites/default/files/standards/documents/DSP0266_1.11.0.pdf)
, it is not clear that the events need to be persisted.I will ask in the
DMTF for the persistence of the events.
>> We have two options:
>> 1) Each transport interface listens for the Dbus signals and write
>> it to their app specific file.
>> 2) Each openbmc repo must use log::report for each D-bus property
>> update/ interface added.
> #2 is absolutely unworkable on the surface to me. log::report is to
> create a error entry (xyz.openbmc_project.Logging.Entry), which creates
> a dbus-object, which would cause log::report to be called, which creates
> a dbus-object, which ...
>
> Even if what you meant was something like logging::log<info>, this seems
> pretty heavy. I'm not sure this is something that can be inserted into
> sdbusplus, especially for the ASIO-based object servers, because in many
> cases applications register their own callback. For the sdbus++
> generated server bindings we could squeeze it in. But, what you're
> proposing here is essentially a "journal-as-dbus-monitor". We'd
> certainly need to make some measurements on how many kB/s worth of
> journal entries this would create because I suspect we could end up
> burning out a NAND flash with as many writes as that would induce.
I would respond on the same once my query gets answered from DMTF.
If my query gets answered yes then we have to write on flash but let's
wait for it,
[-- Attachment #2: Type: text/html, Size: 7343 bytes --]
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-19 13:26 ` Ratan Gupta
@ 2020-06-19 22:19 ` Bills, Jason M
2020-06-23 7:27 ` Ratan Gupta
1 sibling, 0 replies; 44+ messages in thread
From: Bills, Jason M @ 2020-06-19 22:19 UTC (permalink / raw)
To: openbmc
On 6/19/2020 6:26 AM, Ratan Gupta wrote:
>
> On 6/18/20 2:15 AM, Patrick Williams wrote:
>> On Wed, Jun 17, 2020 at 05:38:47PM +0530, Ratan Gupta wrote:
>>> Hi James,Pattrick.
>>>
>>>>> Can't we do this already today by defining a simple errors/metadata file
>>>>> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it?
>>>>> This will create a record on dbus in phosphor-logging.
>>>>>
>>>> I think the original concern was with supporting on the order of
>>>> 10,000 log entries, having this on d-bus seemed impractical. Also the
>>>> free log rotation the journal provides is useful. Now modifying the
>>>> logging::report<...> to conditionally log to the journal seems realistic.
>>> My intention was not to re-implement the logging, my intention was to
>>> extend/use the existing design which we already have it below.
>>>
>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md
>>>
>>> I was trying not to bring the Redfish specific stuff in each individual
>>> repo, instead each transport can listen for
>>> Dbus events and write to the journal which goes to their app specific file.
>> Good. This wasn't clear from the earlier email. Thanks.
>>
>>
>>> As we are in agreement that we want to use the journal for persistence
>>> and log rotate feature.
>> I'm not convinced there is agreement on this. There has been
>> disagreement about even using the journal for phosphor-logging use since
>> the beginning and I suspect there would be less agreement on another
>> application using it as its own IPC mechanism.
>>
>> Just because a hammer can be used to insert a nail into a board doesn't
>> mean you use it insert any pointy thing into a flat thing. [ Just
>> because the journal provides log rotation and persistance doesn't mean
>> you should use it for every feature needing log rotation and
>> persistance. ]
>>
>>
>>> ***** As per the Redfish one of the requirement is we need the log for
>>> most of the Dbus Property update/interface added as they
>>> are mapped to some Redfish Resource and the bmcweb has to send the
>>> Resource updated/modified signal to the
>>> Redfish client. ******
>
> Jaosn: You asked the following query in other thread /*"Why do we want
> to log on D-Bus property updates? This seems like it will be too noisy
> for the EventLog*"/
>
> Eg: Client is interested for an event when ever there is any user
> add/delete or network configuration change or there is a log entry
> resource gets created,To handle this request the flow would be
My understanding is for the first iteration we are only providing
notifications for event messages. So, if it isn't in the message
registry, then you cannot register for that event?
It seems like the full resource change event notification will be a
major feature and should probably get a dedicated design.
>
> Redfish Client subscribe for "ResourceType" eg:
> "EthernetService,AccountService,LogService" with subordinate resources
> property as truewhich means the Client is looking for updates on the
> subscribed resources and the subordinates resource, These redfish
> resources(EthernetInterface, IP address, ManagerAccount, AccountService)
> would be mapped to some D-bus Resources, hence some application/bmcweb
> would monitor the Dbus signals on the interested Dbus objects and send
> the Redfish event to the subscribed client.
>
> Apparo: Please correct me if I am missing something.
>
>> I don't know Redfish well, so bear with me if there is something obvious
>> I'm missing. But, the first part of this "requirement" doesn't seem to
>> follow from the second part of the "requirement" to me.
>>
>> Sending a signal of a property changing to the Redfish clients is
>> straight-forwawrd; Redfish should subscribe to all the appropriate
>> dbus-events. I don't understand how this implies any sort of logging.
>> Where does the logging part of this requirement come from?
> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749/16/designs/redfish-eventservice.md#474
>
> While I am reading the redfish
> spec(https://www.dmtf.org/sites/default/files/standards/documents/DSP0266_1.11.0.pdf)
> , it is not clear that the events need to be persisted.I will ask in the
> DMTF for the persistence of the events.
>
>>> We have two options:
>>> 1) Each transport interface listens for the Dbus signals and write
>>> it to their app specific file.
>>> 2) Each openbmc repo must use log::report for each D-bus property
>>> update/ interface added.
>> #2 is absolutely unworkable on the surface to me. log::report is to
>> create a error entry (xyz.openbmc_project.Logging.Entry), which creates
>> a dbus-object, which would cause log::report to be called, which creates
>> a dbus-object, which ...
>>
>> Even if what you meant was something like logging::log<info>, this seems
>> pretty heavy. I'm not sure this is something that can be inserted into
>> sdbusplus, especially for the ASIO-based object servers, because in many
>> cases applications register their own callback. For the sdbus++
>> generated server bindings we could squeeze it in. But, what you're
>> proposing here is essentially a "journal-as-dbus-monitor". We'd
>> certainly need to make some measurements on how many kB/s worth of
>> journal entries this would create because I suspect we could end up
>> burning out a NAND flash with as many writes as that would induce.
> I would respond on the same once my query gets answered from DMTF.
> If my query gets answered yes then we have to write on flash but let's
> wait for it,
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-19 13:26 ` Ratan Gupta
2020-06-19 22:19 ` Bills, Jason M
@ 2020-06-23 7:27 ` Ratan Gupta
2020-06-24 16:24 ` James Feist
2020-06-25 17:26 ` Brad Bishop
1 sibling, 2 replies; 44+ messages in thread
From: Ratan Gupta @ 2020-06-23 7:27 UTC (permalink / raw)
To: Patrick Williams
Cc: Puli, Apparao, Bills, Jason M, Brad Bishop, James Feist, openbmc
[-- Attachment #1: Type: text/plain, Size: 6517 bytes --]
On 6/19/20 6:56 PM, Ratan Gupta wrote:
>
>
> On 6/18/20 2:15 AM, Patrick Williams wrote:
>> On Wed, Jun 17, 2020 at 05:38:47PM +0530, Ratan Gupta wrote:
>>> Hi James,Pattrick.
>>>
>>>>> Can't we do this already today by defining a simple errors/metadata file
>>>>> in phosphor-dbus-interfaces and calling 'logging::report<...>' on it?
>>>>> This will create a record on dbus in phosphor-logging.
>>>>>
>>>> I think the original concern was with supporting on the order of
>>>> 10,000 log entries, having this on d-bus seemed impractical. Also the
>>>> free log rotation the journal provides is useful. Now modifying the
>>>> logging::report<...> to conditionally log to the journal seems realistic.
>>> My intention was not to re-implement the logging, my intention was to
>>> extend/use the existing design which we already have it below.
>>>
>>> https://github.com/openbmc/docs/blob/master/architecture/redfish-logging-in-bmcweb.md
>>>
>>> I was trying not to bring the Redfish specific stuff in each individual
>>> repo, instead each transport can listen for
>>> Dbus events and write to the journal which goes to their app specific file.
>> Good. This wasn't clear from the earlier email. Thanks.
>>
>>
>>> As we are in agreement that we want to use the journal for persistence
>>> and log rotate feature.
>> I'm not convinced there is agreement on this. There has been
>> disagreement about even using the journal for phosphor-logging use since
>> the beginning and I suspect there would be less agreement on another
>> application using it as its own IPC mechanism.
>>
>> Just because a hammer can be used to insert a nail into a board doesn't
>> mean you use it insert any pointy thing into a flat thing. [ Just
>> because the journal provides log rotation and persistance doesn't mean
>> you should use it for every feature needing log rotation and
>> persistance. ]
>>
>>
>>> ***** As per the Redfish one of the requirement is we need the log for
>>> most of the Dbus Property update/interface added as they
>>> are mapped to some Redfish Resource and the bmcweb has to send the
>>> Resource updated/modified signal to the
>>> Redfish client. ******
>
> Jaosn: You asked the following query in other thread /*"Why do we want
> to log on D-Bus property updates? This seems like it will be too
> noisy for the EventLog*"/
>
> Eg: Client is interested for an event when ever there is any user
> add/delete or network configuration change or there is a log entry
> resource gets created,To handle this request the flow would be
>
> Redfish Client subscribe for "ResourceType" eg:
> "EthernetService,AccountService,LogService" with subordinate
> resources property as truewhich means the Client is looking for
> updates on the subscribed resources and the subordinates resource,
> These redfish resources(EthernetInterface, IP address, ManagerAccount,
> AccountService) would be mapped to some D-bus Resources, hence some
> application/bmcweb would monitor the Dbus signals on the interested
> Dbus objects and send the Redfish event to the subscribed client.
>
> Apparo: Please correct me if I am missing something.
>
>> I don't know Redfish well, so bear with me if there is something obvious
>> I'm missing. But, the first part of this "requirement" doesn't seem to
>> follow from the second part of the "requirement" to me.
>>
>> Sending a signal of a property changing to the Redfish clients is
>> straight-forwawrd; Redfish should subscribe to all the appropriate
>> dbus-events. I don't understand how this implies any sort of logging.
>> Where does the logging part of this requirement come from?
> https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/24749/16/designs/redfish-eventservice.md#474
>
> While I am reading the redfish
> spec(https://www.dmtf.org/sites/default/files/standards/documents/DSP0266_1.11.0.pdf)
> , it is not clear that the events need to be persisted.I will ask in
> the DMTF for the persistence of the events.
I have got an update from the DMTF that persistence is not required to
be implemented for the
events(https://redfish.dmtf.org/schemas/v1/Event.v1_5_0.json),
which implies we don't need to log the events in Journal.
More info(https://github.com/DMTF/Redfish/issues/3646)
Now updating the proposal where redfish client subscribed for Resource
Type based events
1) Client would subscribe for the Redfish Resource(eg:ManagerAccount) to
receive events like Account add/delete/modify
Hence need for mapping from (RedfishResource to Dbus Resource)
2) Have the mapping info from Redfish resources to DBUS Resources (Some
JSon file may have this mapping)
2) Have the reverse mapping from DBUS Resources to Redfish Resources
* This is needed to send the Redfish event if there is any changes
in the corresponding D-bus resources.
eg BMC state change/network change etc.
3) bmcweb would monitor the DBUS events
4) Get the Redfish Path from the Mapping(2) and send the Redfish event
5) Bmcweb would buffer N number of events that can be configurable by
redfish client. Buffer would get cleaned up in case of bmcweb restart or
BMC reset.
James, Apparao: Please let me know if there is any more concern with
this approach.
>
>>> We have two options:
>>> 1) Each transport interface listens for the Dbus signals and write
>>> it to their app specific file.
>>> 2) Each openbmc repo must use log::report for each D-bus property
>>> update/ interface added.
>> #2 is absolutely unworkable on the surface to me. log::report is to
>> create a error entry (xyz.openbmc_project.Logging.Entry), which creates
>> a dbus-object, which would cause log::report to be called, which creates
>> a dbus-object, which ...
>>
>> Even if what you meant was something like logging::log<info>, this seems
>> pretty heavy. I'm not sure this is something that can be inserted into
>> sdbusplus, especially for the ASIO-based object servers, because in many
>> cases applications register their own callback. For the sdbus++
>> generated server bindings we could squeeze it in. But, what you're
>> proposing here is essentially a "journal-as-dbus-monitor". We'd
>> certainly need to make some measurements on how many kB/s worth of
>> journal entries this would create because I suspect we could end up
>> burning out a NAND flash with as many writes as that would induce.
> I would respond on the same once my query gets answered from DMTF.
> If my query gets answered yes then we have to write on flash but let's
> wait for it,
[-- Attachment #2: Type: text/html, Size: 9634 bytes --]
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-23 7:27 ` Ratan Gupta
@ 2020-06-24 16:24 ` James Feist
2020-06-24 20:39 ` Patrick Williams
` (2 more replies)
2020-06-25 17:26 ` Brad Bishop
1 sibling, 3 replies; 44+ messages in thread
From: James Feist @ 2020-06-24 16:24 UTC (permalink / raw)
To: Ratan Gupta, Patrick Williams
Cc: Puli, Apparao, Bills, Jason M, Brad Bishop, openbmc
> I have got an update from the DMTF that persistence is not required to
> be implemented for the
> events(https://redfish.dmtf.org/schemas/v1/Event.v1_5_0.json),
> which implies we don't need to log the events in Journal.
>
> More info(https://github.com/DMTF/Redfish/issues/3646)
>
> Now updating the proposal where redfish client subscribed for Resource
> Type based events
> 1) Client would subscribe for the Redfish Resource(eg:ManagerAccount) to
> receive events like Account add/delete/modify
> Hence need for mapping from (RedfishResource to Dbus Resource)
>
> 2) Have the mapping info from Redfish resources to DBUS Resources (Some
> JSon file may have this mapping)
This assumes the mapping is static, which on many systems it isn't,
right? I think this needs to be developed to see what it would be like.
>
> 2) Have the reverse mapping from DBUS Resources to Redfish Resources
> * This is needed to send the Redfish event if there is any changes
> in the corresponding D-bus resources.
> eg BMC state change/network change etc.
>
> 3) bmcweb would monitor the DBUS events
Which events specifically? It seems like it would be monitoring lots of
events in this proposal. Some examples could be useful.
>
> 4) Get the Redfish Path from the Mapping(2) and send the Redfish event
>
> 5) Bmcweb would buffer N number of events that can be configurable by
> redfish client. Buffer would get cleaned up in case of bmcweb restart or
> BMC reset.
>
Can you please push a patch to the event service design doc with the
change to the design you'd like? Some examples in the design would help
with understanding the proposal. This thread is quite long and hard to
follow for someone not involved, it'd be good to document the proposed
changes.
https://github.com/openbmc/docs/blob/master/designs/redfish-eventservice.md
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-24 16:24 ` James Feist
@ 2020-06-24 20:39 ` Patrick Williams
2020-06-25 13:45 ` Brad Bishop
2020-06-25 15:49 ` Brad Bishop
2 siblings, 0 replies; 44+ messages in thread
From: Patrick Williams @ 2020-06-24 20:39 UTC (permalink / raw)
To: James Feist
Cc: Ratan Gupta, Puli, Apparao, Bills, Jason M, Brad Bishop, openbmc
[-- Attachment #1: Type: text/plain, Size: 591 bytes --]
On Wed, Jun 24, 2020 at 09:24:03AM -0700, James Feist wrote:
> > 2) Have the mapping info from Redfish resources to DBUS Resources (Some
> > JSon file may have this mapping)
>
> This assumes the mapping is static, which on many systems it isn't,
> right? I think this needs to be developed to see what it would be like.
>
Don't we, in effect, have the mapping from Redfish to dbus by nature of
the Redfish providers that create their content from dbus objects? Is
there any way to reuse this information to create the corresponding
Redfish events?
--
Patrick Williams
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-24 16:24 ` James Feist
2020-06-24 20:39 ` Patrick Williams
@ 2020-06-25 13:45 ` Brad Bishop
2020-06-25 16:42 ` James Feist
2020-06-25 15:49 ` Brad Bishop
2 siblings, 1 reply; 44+ messages in thread
From: Brad Bishop @ 2020-06-25 13:45 UTC (permalink / raw)
To: James Feist, Ratan Gupta, Patrick Williams
Cc: Bills, Jason M, Puli, Apparao, openbmc
On Wed, 2020-06-24 at 09:24 -0700, James Feist wrote:
> > 3) bmcweb would monitor the DBUS events
>
> Which events specifically? It seems like it would be monitoring lots
> of events in this proposal.
Just curious what is the concern around monitoring lots of events?
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-24 16:24 ` James Feist
2020-06-24 20:39 ` Patrick Williams
2020-06-25 13:45 ` Brad Bishop
@ 2020-06-25 15:49 ` Brad Bishop
2020-06-25 16:44 ` James Feist
2 siblings, 1 reply; 44+ messages in thread
From: Brad Bishop @ 2020-06-25 15:49 UTC (permalink / raw)
To: James Feist, Ratan Gupta, Patrick Williams
Cc: Bills, Jason M, Puli, Apparao, openbmc
On Wed, 2020-06-24 at 09:24 -0700, James Feist wrote:
> This thread is quite long and hard to follow for someone not involved,
> it'd be good to document the proposed changes.
You are right of course that this thread has become hard to follow. I
think careful selection of reply context really would have helped.
FWIW I have asked the team here at IBM to hash out designs to some level
of consensus here on the list before making design submissions to gerrit
because IMO reviews of half baked designs in gerrit are even harder to
follow than email threads like this one 🙂
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-25 13:45 ` Brad Bishop
@ 2020-06-25 16:42 ` James Feist
0 siblings, 0 replies; 44+ messages in thread
From: James Feist @ 2020-06-25 16:42 UTC (permalink / raw)
To: Brad Bishop, Ratan Gupta, Patrick Williams
Cc: Bills, Jason M, Puli, Apparao, openbmc
On 6/25/2020 6:45 AM, Brad Bishop wrote:
> On Wed, 2020-06-24 at 09:24 -0700, James Feist wrote:
>>> 3) bmcweb would monitor the DBUS events
>>
>> Which events specifically? It seems like it would be monitoring lots
>> of events in this proposal.
>
> Just curious what is the concern around monitoring lots of events?
>
Performance as always, depending on the number of matches needed.
Although a POC could prove this not an issue.
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-25 15:49 ` Brad Bishop
@ 2020-06-25 16:44 ` James Feist
0 siblings, 0 replies; 44+ messages in thread
From: James Feist @ 2020-06-25 16:44 UTC (permalink / raw)
To: Brad Bishop, Ratan Gupta, Patrick Williams
Cc: Bills, Jason M, Puli, Apparao, openbmc
On 6/25/2020 8:49 AM, Brad Bishop wrote:
> On Wed, 2020-06-24 at 09:24 -0700, James Feist wrote:
>> This thread is quite long and hard to follow for someone not involved,
>> it'd be good to document the proposed changes.
>
> You are right of course that this thread has become hard to follow. I
> think careful selection of reply context really would have helped.
>
> FWIW I have asked the team here at IBM to hash out designs to some level
> of consensus here on the list before making design submissions to gerrit
> because IMO reviews of half baked designs in gerrit are even harder to
> follow than email threads like this one 🙂
>
That's good. Was just hoping that the 'final design' wasn't this email,
as it would be difficult to see we matched a design.
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-23 7:27 ` Ratan Gupta
2020-06-24 16:24 ` James Feist
@ 2020-06-25 17:26 ` Brad Bishop
2020-06-25 18:55 ` Bills, Jason M
1 sibling, 1 reply; 44+ messages in thread
From: Brad Bishop @ 2020-06-25 17:26 UTC (permalink / raw)
To: Ratan Gupta, Patrick Williams
Cc: Bills, Jason M, Puli, Apparao, James Feist, openbmc
On Tue, 2020-06-23 at 12:57 +0530, Ratan Gupta wrote:
> 1) Client would subscribe for the Redfish Resource(eg:ManagerAccount)
> to receive events like Account add/delete/modify. Hence need for
> mapping from (RedfishResource to Dbus Resource)
>
> 2) Have the mapping info from Redfish resources to DBUS Resources
> (Some JSon file may have this mapping)
It probably doesn't make sense to have any json files. The application
logic itself does the mapping.
> 3) Have the reverse mapping from DBUS Resources to Redfish Resources
> This is needed to send the Redfish event if there is any changes in
> the corresponding D-bus resources. eg BMC state change/network change
> etc.
>
> 4) bmcweb would monitor the DBUS events
>
> 5) Get the Redfish Path from the Mapping(3) and send the Redfish event
Let me try to restate 3-5. We already have application logic in bmcweb
that maps redfish resources to dbus interfaces, triggered by redfish
client requests (like a GET or PATCH). The proposal here I think is to
implement the reverse - add application logic in bmcweb that maps dbus
interfaces to redfish resources, triggered by dbus signals. I think
this is more or less what was suggested by Patrick?
> Don't we, in effect, have the mapping from Redfish to dbus by nature
> of the Redfish providers that create their content from dbus objects?
> James, Apparao: Please let me know if there is any more concern with
> this approach.
Ratan you had 2x #2s in the flow above, which I fixed. Please let me
know if I got it wrong.
The main concerns with this approach that I have heard are too many
signal matches and complexity of the logic implementing the reverse map.
I can definitely see how the logic will get complicated given our
current dbus data model.
One idea floating around to address these is inventing a journal
metadata scheme that is management interface agnostic. I understand the
motivation behind that - it is much simpler for an application to slide
a single journal entry into the journal with all the metadata needed to
generate events, than it is for an application to snoop multiple signals
off dbus and pull events out of that.
But I wonder if inventing a management interface agnostic scheme for
adding events to the journal is really just inventing a new data model
for how we represent things in a server - e.g. are we just working
around our dbus data model? Maybe we should fix it instead, so that it
isn't so difficult for applications to use it? With that said I don't
know how to do this and I could use more concrete details on which areas
of the data model make it hard to consume signals. Does anyone have any
ideas or examples?
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-25 17:26 ` Brad Bishop
@ 2020-06-25 18:55 ` Bills, Jason M
2020-06-29 8:00 ` Ratan Gupta
2020-06-29 8:07 ` Ratan Gupta
0 siblings, 2 replies; 44+ messages in thread
From: Bills, Jason M @ 2020-06-25 18:55 UTC (permalink / raw)
To: openbmc
On 6/25/2020 10:26 AM, Brad Bishop wrote:
> One idea floating around to address these is inventing a journal
> metadata scheme that is management interface agnostic. I understand the
> motivation behind that - it is much simpler for an application to slide
> a single journal entry into the journal with all the metadata needed to
> generate events, than it is for an application to snoop multiple signals
> off dbus and pull events out of that.
>
> But I wonder if inventing a management interface agnostic scheme for
> adding events to the journal is really just inventing a new data model
> for how we represent things in a server - e.g. are we just working
> around our dbus data model? Maybe we should fix it instead, so that it
> isn't so difficult for applications to use it? With that said I don't
> know how to do this and I could use more concrete details on which areas
> of the data model make it hard to consume signals. Does anyone have any
> ideas or examples?
>
On this, I think we may want to separate logging vs. eventing both in
this feature discussion and in the tools we want to use.
When we were talking about logging, I think the journal made sense since
it is designed for logging and has benefits around that usage. However,
I agree that it doesn't seem like the right tool for sending and
receiving events and signals and that D-Bus sounds like the right tool
for that.
I think I'm still a little confused at the scope. My understanding was
that this initial design for EventService was only for monitoring event
messages and not resources in general. It seems like it may not make
sense to try to use the same tools and approach for both message
monitoring and resource monitoring? Do we need to treat them separately
for now to simplify the discussion?
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-25 18:55 ` Bills, Jason M
@ 2020-06-29 8:00 ` Ratan Gupta
2020-06-29 8:07 ` Ratan Gupta
1 sibling, 0 replies; 44+ messages in thread
From: Ratan Gupta @ 2020-06-29 8:00 UTC (permalink / raw)
To: Bills, Jason M, openbmc
Hi Jason,james
On 6/26/20 12:25 AM, Bills, Jason M wrote:
>
>
> On 6/25/2020 10:26 AM, Brad Bishop wrote:
>> One idea floating around to address these is inventing a journal
>> metadata scheme that is management interface agnostic. I understand the
>> motivation behind that - it is much simpler for an application to slide
>> a single journal entry into the journal with all the metadata needed to
>> generate events, than it is for an application to snoop multiple signals
>> off dbus and pull events out of that.
>>
>> But I wonder if inventing a management interface agnostic scheme for
>> adding events to the journal is really just inventing a new data model
>> for how we represent things in a server - e.g. are we just working
>> around our dbus data model? Maybe we should fix it instead, so that it
>> isn't so difficult for applications to use it? With that said I don't
>> know how to do this and I could use more concrete details on which areas
>> of the data model make it hard to consume signals. Does anyone have any
>> ideas or examples?
>>
>
> On this, I think we may want to separate logging vs. eventing both in
> this feature discussion and in the tools we want to use.
>
> When we were talking about logging, I think the journal made sense
> since it is designed for logging and has benefits around that usage.
> However, I agree that it doesn't seem like the right tool for sending
> and receiving events and signals and that D-Bus sounds like the right
> tool for that.
>
> I think I'm still a little confused at the scope. My understanding
> was that this initial design for EventService was only for monitoring
> event messages and not resources in general. It seems like it may not
> make sense to try to use the same tools and approach for both message
> monitoring and resource monitoring? Do we need to treat them
> separately for now to simplify the discussion?
Jason, When you say event messages? What do you mean, Do you mean to say
"/redfish/v1/Systems/system/Logservices/eventlog"?
If yes then this should also go as Resource Event, When ever any log
entry gets created under System Log
(/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would
notify to the Redfish client saying that "ResourceCreated" with the URL
of the Resource.
After recieving this event Redfish client will do a GET request on the
URL(retrieved as part of event) to get the content of the log.
I would be coming up with few design approaches and downside with each
approach to take it to conclusion.
Ratan
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-25 18:55 ` Bills, Jason M
2020-06-29 8:00 ` Ratan Gupta
@ 2020-06-29 8:07 ` Ratan Gupta
2020-06-29 15:33 ` Bills, Jason M
1 sibling, 1 reply; 44+ messages in thread
From: Ratan Gupta @ 2020-06-29 8:07 UTC (permalink / raw)
To: Bills, Jason M, openbmc
On 6/26/20 12:25 AM, Bills, Jason M wrote:
>
>
> On 6/25/2020 10:26 AM, Brad Bishop wrote:
>> One idea floating around to address these is inventing a journal
>> metadata scheme that is management interface agnostic. I understand the
>> motivation behind that - it is much simpler for an application to slide
>> a single journal entry into the journal with all the metadata needed to
>> generate events, than it is for an application to snoop multiple signals
>> off dbus and pull events out of that.
>>
>> But I wonder if inventing a management interface agnostic scheme for
>> adding events to the journal is really just inventing a new data model
>> for how we represent things in a server - e.g. are we just working
>> around our dbus data model? Maybe we should fix it instead, so that it
>> isn't so difficult for applications to use it? With that said I don't
>> know how to do this and I could use more concrete details on which areas
>> of the data model make it hard to consume signals. Does anyone have any
>> ideas or examples?
>>
>
> On this, I think we may want to separate logging vs. eventing both in
> this feature discussion and in the tools we want to use.
>
> When we were talking about logging, I think the journal made sense
> since it is designed for logging and has benefits around that usage.
> However, I agree that it doesn't seem like the right tool for sending
> and receiving events and signals and that D-Bus sounds like the right
> tool for that.
>
> I think I'm still a little confused at the scope. My understanding
> was that this initial design for EventService was only for monitoring
> event messages and not resources in general. It seems like it may not
> make sense to try to use the same tools and approach for both message
> monitoring and resource monitoring? Do we need to treat them
> separately for now to simplify the discussion?
Jason, When you say event messages? What do you mean, Do you mean to say
"/redfish/v1/Systems/system/Logservices/eventlog"?
If yes then this should also go as Resource Event, When ever any log
entry gets created under System Log
(/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would
notify to the Redfish client saying that "ResourceCreated" with the URL
of the Resource.
After receiving this event Redfish client will do a GET request on the
URL(retrieved as part of event) to get the content of the log.
This will become generic infra for all types of events.
I would be coming up with few design approaches and downside with each
approach to take it to conclusion.
Ratan
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-29 8:07 ` Ratan Gupta
@ 2020-06-29 15:33 ` Bills, Jason M
2020-07-03 10:15 ` Ratan Gupta
0 siblings, 1 reply; 44+ messages in thread
From: Bills, Jason M @ 2020-06-29 15:33 UTC (permalink / raw)
To: openbmc
On 6/29/2020 1:07 AM, Ratan Gupta wrote:
>
> On 6/26/20 12:25 AM, Bills, Jason M wrote:
>>
>>
>> On 6/25/2020 10:26 AM, Brad Bishop wrote:
>>> One idea floating around to address these is inventing a journal
>>> metadata scheme that is management interface agnostic. I understand the
>>> motivation behind that - it is much simpler for an application to slide
>>> a single journal entry into the journal with all the metadata needed to
>>> generate events, than it is for an application to snoop multiple signals
>>> off dbus and pull events out of that.
>>>
>>> But I wonder if inventing a management interface agnostic scheme for
>>> adding events to the journal is really just inventing a new data model
>>> for how we represent things in a server - e.g. are we just working
>>> around our dbus data model? Maybe we should fix it instead, so that it
>>> isn't so difficult for applications to use it? With that said I don't
>>> know how to do this and I could use more concrete details on which areas
>>> of the data model make it hard to consume signals. Does anyone have any
>>> ideas or examples?
>>>
>>
>> On this, I think we may want to separate logging vs. eventing both in
>> this feature discussion and in the tools we want to use.
>>
>> When we were talking about logging, I think the journal made sense
>> since it is designed for logging and has benefits around that usage.
>> However, I agree that it doesn't seem like the right tool for sending
>> and receiving events and signals and that D-Bus sounds like the right
>> tool for that.
>>
>> I think I'm still a little confused at the scope. My understanding
>> was that this initial design for EventService was only for monitoring
>> event messages and not resources in general. It seems like it may not
>> make sense to try to use the same tools and approach for both message
>> monitoring and resource monitoring? Do we need to treat them
>> separately for now to simplify the discussion?
> Jason, When you say event messages? What do you mean, Do you mean to say
> "/redfish/v1/Systems/system/Logservices/eventlog"? >
> If yes then this should also go as Resource Event, When ever any log
> entry gets created under System Log
> (/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would
> notify to the Redfish client saying that "ResourceCreated" with the URL
> of the Resource.
Yes, new entries under
"/redfish/v1/Systems/system/Logservices/eventlog", but I thought you
could register for specific MessageIDs, so it's not just a generic "new
resource" event like others would be.
>
> After receiving this event Redfish client will do a GET request on the
> URL(retrieved as part of event) to get the content of the log.
>
> This will become generic infra for all types of events.
What I'm saying is I don't know if there is a good generic solution to
cover both the EventLog and all other resources. I believe the current
EventService implementation was designed only for EventLog and may not
work well for generic resource events.
>
> I would be coming up with few design approaches and downside with each
> approach to take it to conclusion.
Thanks! What I'm proposing is that we clarify or possibly separate the
discussions about EventLog vs. generic resources to avoid confusion and
come up with the right solutions for each.
>
> Ratan
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-06-29 15:33 ` Bills, Jason M
@ 2020-07-03 10:15 ` Ratan Gupta
2020-07-06 21:30 ` Bills, Jason M
0 siblings, 1 reply; 44+ messages in thread
From: Ratan Gupta @ 2020-07-03 10:15 UTC (permalink / raw)
To: openbmc
On 6/29/20 9:03 PM, Bills, Jason M wrote:
>
>
> On 6/29/2020 1:07 AM, Ratan Gupta wrote:
>>
>> On 6/26/20 12:25 AM, Bills, Jason M wrote:
>>>
>>>
>>> On 6/25/2020 10:26 AM, Brad Bishop wrote:
>>>> One idea floating around to address these is inventing a journal
>>>> metadata scheme that is management interface agnostic. I
>>>> understand the
>>>> motivation behind that - it is much simpler for an application to
>>>> slide
>>>> a single journal entry into the journal with all the metadata
>>>> needed to
>>>> generate events, than it is for an application to snoop multiple
>>>> signals
>>>> off dbus and pull events out of that.
>>>>
>>>> But I wonder if inventing a management interface agnostic scheme for
>>>> adding events to the journal is really just inventing a new data model
>>>> for how we represent things in a server - e.g. are we just working
>>>> around our dbus data model? Maybe we should fix it instead, so
>>>> that it
>>>> isn't so difficult for applications to use it? With that said I don't
>>>> know how to do this and I could use more concrete details on which
>>>> areas
>>>> of the data model make it hard to consume signals. Does anyone
>>>> have any
>>>> ideas or examples?
>>>>
>>>
>>> On this, I think we may want to separate logging vs. eventing both
>>> in this feature discussion and in the tools we want to use.
>>>
>>> When we were talking about logging, I think the journal made sense
>>> since it is designed for logging and has benefits around that usage.
>>> However, I agree that it doesn't seem like the right tool for
>>> sending and receiving events and signals and that D-Bus sounds like
>>> the right tool for that.
>>>
>>> I think I'm still a little confused at the scope. My understanding
>>> was that this initial design for EventService was only for
>>> monitoring event messages and not resources in general. It seems
>>> like it may not make sense to try to use the same tools and approach
>>> for both message monitoring and resource monitoring? Do we need to
>>> treat them separately for now to simplify the discussion?
>> Jason, When you say event messages? What do you mean, Do you mean to
>> say "/redfish/v1/Systems/system/Logservices/eventlog"? >
>> If yes then this should also go as Resource Event, When ever any log
>> entry gets created under System Log
>> (/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would
>> notify to the Redfish client saying that "ResourceCreated" with the
>> URL of the Resource.
> Yes, new entries under
> "/redfish/v1/Systems/system/Logservices/eventlog", but I thought you
> could register for specific MessageIDs, so it's not just a generic
> "new resource" event like others would be.
Can we register for MessageID? I thought client can register for whole
registry not a specific Message ID.
>
>
>>
>> After receiving this event Redfish client will do a GET request on
>> the URL(retrieved as part of event) to get the content of the log.
>>
>> This will become generic infra for all types of events.
> What I'm saying is I don't know if there is a good generic solution to
> cover both the EventLog and all other resources. I believe the
> current EventService implementation was designed only for EventLog and
> may not work well for generic resource events.
Can you get me the example payload for EventLog which is going to be
sent with the current design? I am not sure how the eventlog and other
resources are different.
For eventLogs also we have the associated D-bus
objects(/xyz/openbmc_project/logging,/xyz/openbmc_project/dump etc)
>
>>
>> I would be coming up with few design approaches and downside with
>> each approach to take it to conclusion.
> Thanks! What I'm proposing is that we clarify or possibly separate
> the discussions about EventLog vs. generic resources to avoid
> confusion and come up with the right solutions for each.
>
>>
>> Ratan
>>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-07-03 10:15 ` Ratan Gupta
@ 2020-07-06 21:30 ` Bills, Jason M
2020-07-13 6:32 ` Ratan Gupta
0 siblings, 1 reply; 44+ messages in thread
From: Bills, Jason M @ 2020-07-06 21:30 UTC (permalink / raw)
To: openbmc
On 7/3/2020 3:15 AM, Ratan Gupta wrote:
>>>> I think I'm still a little confused at the scope. My understanding
>>>> was that this initial design for EventService was only for
>>>> monitoring event messages and not resources in general. It seems
>>>> like it may not make sense to try to use the same tools and approach
>>>> for both message monitoring and resource monitoring? Do we need to
>>>> treat them separately for now to simplify the discussion?
>>> Jason, When you say event messages? What do you mean, Do you mean to
>>> say "/redfish/v1/Systems/system/Logservices/eventlog"? >
>>> If yes then this should also go as Resource Event, When ever any log
>>> entry gets created under System Log
>>> (/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC would
>>> notify to the Redfish client saying that "ResourceCreated" with the
>>> URL of the Resource.
>> Yes, new entries under
>> "/redfish/v1/Systems/system/Logservices/eventlog", but I thought you
>> could register for specific MessageIDs, so it's not just a generic
>> "new resource" event like others would be.
>
> Can we register for MessageID? I thought client can register for whole
> registry not a specific Message ID.
>
I don't really know. I thought that's what the current implementation
allowed, but I don't know for sure if it can or should.
>>
>>
>>>
>>> After receiving this event Redfish client will do a GET request on
>>> the URL(retrieved as part of event) to get the content of the log.
>>>
>>> This will become generic infra for all types of events.
>> What I'm saying is I don't know if there is a good generic solution to
>> cover both the EventLog and all other resources. I believe the
>> current EventService implementation was designed only for EventLog and
>> may not work well for generic resource events.
>
> Can you get me the example payload for EventLog which is going to be
> sent with the current design? I am not sure how the eventlog and other
> resources are different.
>
This is based on the assumption that for a LogService, you can register
for a MessageId. If this is not possible, then they might be treated
the same.
> For eventLogs also we have the associated D-bus
> objects(/xyz/openbmc_project/logging,/xyz/openbmc_project/dump etc)
>
For Intel platforms, we don't use /xyz/openbmc_project/logging, so we
don't have D-Bus objects associated with each EventLog LogEntry. We use
rsyslog to create a file that contains many LogEntries.
However, as an unrelated side-thought: linking logging to
/xyz/openbmc_project/dump made me wonder if there is a possible solution
to the logging issue if we treat /xyz/openbmc_project/logging like
/xyz/openbmc_project/dump and place a pointer to the log in the D-Bus
object instead of the log itself?
>>
>>>
>>> I would be coming up with few design approaches and downside with
>>> each approach to take it to conclusion.
>> Thanks! What I'm proposing is that we clarify or possibly separate
>> the discussions about EventLog vs. generic resources to avoid
>> confusion and come up with the right solutions for each.
>>
>>>
>>> Ratan
>>>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-07-06 21:30 ` Bills, Jason M
@ 2020-07-13 6:32 ` Ratan Gupta
2020-07-14 21:08 ` James Feist
0 siblings, 1 reply; 44+ messages in thread
From: Ratan Gupta @ 2020-07-13 6:32 UTC (permalink / raw)
To: openbmc, James Feist, Jason >> Bills, Jason M
Hi James,
As you asked for the examples in the
thread(https://lists.ozlabs.org/pipermail/openbmc/2020-June/022125.html),
I have created the mapping at the following location.
https://gist.github.com/ratagupt/0aa4da098a60d49af90a7e4a6ea6d5f2
1) Map1: Mapping between redfish resources to Dbus resources
2) Map2: Mapping between redfish resource types to the ineterested Dbus
interfaces
3) Map3: Mapping between Dbus resources to redfish resources
I tried to cover the following scenario in the above mapping.
* Redfish resource is mapped to multiple Dbus Resources
* Redfish Property is mapped to single Dbus property
* Redfish Property(complex property) is mapped to multiple dbus property.
* Same type of Redfish Resources are mapped to different Dbus Resources
* Redfish node url having multiple regex : Yet to take a look on this.
Flow would be like as below
=> In bmcweb each Redfish node represents to a Redfish Resource.
=> Each node will be having it's own mapping between Redfish properties
and the Dbus Resources.
=> Some code on bmcweb will walkthrough on each node during startup ,
get this mapping from each node and generate
two mappings
1) Reverse mapping (Dbus Resource to Redfish Resource)(MAP3) and
2) mapping between Resource Types to the interested Dbus
interfaces(MAP2)
=> To start with we will support few resource types and then scale it up
as needed.
=> Map2 would be used when the Redfish client subscribe for the
ResourceType to get the Dbus mappings.
=> Map3 would be used when the Dbus signal gets generated and need the
Redfish mappings.
=> Once we have all thsese mapping gets generated and loaded into the
memory, bmcweb would start listening
for the interfaces listed in map2.
=> Once any Dbus signal gets generated map3 would be used to get the
Redfish mapping.
Please let me know if you have any concerns with this approach.
Ratan
On 7/7/20 3:00 AM, Bills, Jason M wrote:
>
>
> On 7/3/2020 3:15 AM, Ratan Gupta wrote:
>>>>> I think I'm still a little confused at the scope. My
>>>>> understanding was that this initial design for EventService was
>>>>> only for monitoring event messages and not resources in general.
>>>>> It seems like it may not make sense to try to use the same tools
>>>>> and approach for both message monitoring and resource monitoring?
>>>>> Do we need to treat them separately for now to simplify the
>>>>> discussion?
>>>> Jason, When you say event messages? What do you mean, Do you mean
>>>> to say "/redfish/v1/Systems/system/Logservices/eventlog"? >
>>>> If yes then this should also go as Resource Event, When ever any
>>>> log entry gets created under System Log
>>>> (/redfish/v1/Systems/system/Logservices/eventlog/entries), BMC
>>>> would notify to the Redfish client saying that "ResourceCreated"
>>>> with the URL of the Resource.
>>> Yes, new entries under
>>> "/redfish/v1/Systems/system/Logservices/eventlog", but I thought you
>>> could register for specific MessageIDs, so it's not just a generic
>>> "new resource" event like others would be.
>>
>> Can we register for MessageID? I thought client can register for
>> whole registry not a specific Message ID.
>>
> I don't really know. I thought that's what the current implementation
> allowed, but I don't know for sure if it can or should.
>
>>>
>>>
>>>>
>>>> After receiving this event Redfish client will do a GET request on
>>>> the URL(retrieved as part of event) to get the content of the log.
>>>>
>>>> This will become generic infra for all types of events.
>>> What I'm saying is I don't know if there is a good generic solution
>>> to cover both the EventLog and all other resources. I believe the
>>> current EventService implementation was designed only for EventLog
>>> and may not work well for generic resource events.
>>
>> Can you get me the example payload for EventLog which is going to be
>> sent with the current design? I am not sure how the eventlog and
>> other resources are different.
>>
> This is based on the assumption that for a LogService, you can
> register for a MessageId. If this is not possible, then they might be
> treated the same.
>
>> For eventLogs also we have the associated D-bus
>> objects(/xyz/openbmc_project/logging,/xyz/openbmc_project/dump etc)
>>
> For Intel platforms, we don't use /xyz/openbmc_project/logging, so we
> don't have D-Bus objects associated with each EventLog LogEntry. We
> use rsyslog to create a file that contains many LogEntries.
>
> However, as an unrelated side-thought: linking logging to
> /xyz/openbmc_project/dump made me wonder if there is a possible
> solution to the logging issue if we treat /xyz/openbmc_project/logging
> like /xyz/openbmc_project/dump and place a pointer to the log in the
> D-Bus object instead of the log itself?
>
>>>
>>>>
>>>> I would be coming up with few design approaches and downside with
>>>> each approach to take it to conclusion.
>>> Thanks! What I'm proposing is that we clarify or possibly separate
>>> the discussions about EventLog vs. generic resources to avoid
>>> confusion and come up with the right solutions for each.
>>>
>>>>
>>>> Ratan
>>>>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-07-13 6:32 ` Ratan Gupta
@ 2020-07-14 21:08 ` James Feist
2020-07-30 9:10 ` Ratan Gupta
0 siblings, 1 reply; 44+ messages in thread
From: James Feist @ 2020-07-14 21:08 UTC (permalink / raw)
To: Ratan Gupta, openbmc, Jason >> Bills, Jason M, Puli, Apparao
On 7/12/2020 11:32 PM, Ratan Gupta wrote:
> Hi James,
>
> As you asked for the examples in the
> thread(https://lists.ozlabs.org/pipermail/openbmc/2020-June/022125.html), I
> have created the mapping at the following location.
>
> https://gist.github.com/ratagupt/0aa4da098a60d49af90a7e4a6ea6d5f2
Thanks for the examples.
>
> 1) Map1: Mapping between redfish resources to Dbus resources
How does this work when the mapping isn't 1:1 vs D-Bus? Most of the time
the enums to not match the d-bus enum, or take multiple d-bus interface
to distinguish what the value should be. Also how does this work for
discovered things, like multiple chassis?
> 2) Map2: Mapping between redfish resource types to the ineterested Dbus
> interfaces
> 3) Map3: Mapping between Dbus resources to redfish resources
>
>
> I tried to cover the following scenario in the above mapping.
>
> * Redfish resource is mapped to multiple Dbus Resources
> * Redfish Property is mapped to single Dbus property
> * Redfish Property(complex property) is mapped to multiple dbus property.
> * Same type of Redfish Resources are mapped to different Dbus Resources
> * Redfish node url having multiple regex : Yet to take a look on this.
> >
> Flow would be like as below
>
> => In bmcweb each Redfish node represents to a Redfish Resource.
> => Each node will be having it's own mapping between Redfish properties
> and the Dbus Resources.
>
> => Some code on bmcweb will walkthrough on each node during startup ,
> get this mapping from each node and generate
> two mappings
> 1) Reverse mapping (Dbus Resource to Redfish Resource)(MAP3) and
> 2) mapping between Resource Types to the interested Dbus
> interfaces(MAP2)
>
> => To start with we will support few resource types and then scale it up
> as needed.
I think we need an idea of what the final solution will look like for
more complicated properties, or we'll be creating something that isn't
future proof.
>
> => Map2 would be used when the Redfish client subscribe for the
> ResourceType to get the Dbus mappings.
>
> => Map3 would be used when the Dbus signal gets generated and need the
> Redfish mappings.
Why can't these be the same mapping? I think having 3 different maps
makes this very confusing. I also think this is attempting to generalize
the problem too early. If you look at the Redfish code to determine some
of the more complicated properties, sometimes it takes quite a bit of
logic. That logic also would possibly be nice to reuse. Maybe we can
take your idea of a match with a callback to some of the already
existing property parsing? Obviously it would need some cleaning up, but
I could see something with a map of schemas/properties to function
pointers for property parsing.
>
> => Once we have all thsese mapping gets generated and loaded into the
> memory, bmcweb would start listening
> for the interfaces listed in map2.
Why aren't these compiled in? I don't see why they need to be loaded,
could just be in code. Also, they shouldn't be added as matches until
there is a subscriber, or we'll add many unneeded matches.
>
> => Once any Dbus signal gets generated map3 would be used to get the
> Redfish mapping.
>
> Please let me know if you have any concerns with this approach.
>
> Ratan
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Redfish EventService Implementation
2020-07-14 21:08 ` James Feist
@ 2020-07-30 9:10 ` Ratan Gupta
0 siblings, 0 replies; 44+ messages in thread
From: Ratan Gupta @ 2020-07-30 9:10 UTC (permalink / raw)
To: openbmc
Hi James,
On 7/15/20 2:38 AM, James Feist wrote:
> On 7/12/2020 11:32 PM, Ratan Gupta wrote:
>> Hi James,
>>
>> As you asked for the examples in the
>> thread(https://lists.ozlabs.org/pipermail/openbmc/2020-June/022125.html),
>> I have created the mapping at the following location.
>>
>> https://gist.github.com/ratagupt/0aa4da098a60d49af90a7e4a6ea6d5f2
>
> Thanks for the examples.
>
>>
>> 1) Map1: Mapping between redfish resources to Dbus resources
>
> How does this work when the mapping isn't 1:1 vs D-Bus? Most of the
> time the enums to not match the d-bus enum, or take multiple d-bus
> interface to distinguish what the value should be. Also how does this
> work for discovered things, like multiple chassis?
I have not mentioned in the below use cases, but just look at the url
https://gist.github.com/ratagupt/0aa4da098a60d49af90a7e4a6ea6d5f2#file-gistfile1-txt-L77
<https://gist.github.com/ratagupt/0aa4da098a60d49af90a7e4a6ea6d5f2#file-gistfile1-txt-L77>
There we are covering the resources which are getting added dynamically.
>
>
>> 2) Map2: Mapping between redfish resource types to the ineterested
>> Dbus interfaces
>> 3) Map3: Mapping between Dbus resources to redfish resources
>>
>>
>> I tried to cover the following scenario in the above mapping.
>>
>> * Redfish resource is mapped to multiple Dbus Resources
>> * Redfish Property is mapped to single Dbus property
>> * Redfish Property(complex property) is mapped to multiple dbus
>> property.
>> * Same type of Redfish Resources are mapped to different Dbus Resources
>> * Redfish node url having multiple regex : Yet to take a look on this.
>> >
>> Flow would be like as below
>>
>> => In bmcweb each Redfish node represents to a Redfish Resource.
>> => Each node will be having it's own mapping between Redfish
>> properties and the Dbus Resources.
>>
>> => Some code on bmcweb will walkthrough on each node during startup ,
>> get this mapping from each node and generate
>> two mappings
>> 1) Reverse mapping (Dbus Resource to Redfish Resource)(MAP3) and
>> 2) mapping between Resource Types to the interested Dbus
>> interfaces(MAP2)
>>
>> => To start with we will support few resource types and then scale it
>> up as needed.
>
> I think we need an idea of what the final solution will look like for
> more complicated properties, or we'll be creating something that isn't
> future proof.
I thought I covered the complicated one in the examples(gist link above)
but just let me know the other redfish resource which I can take it in
the examples.
>
>>
>> => Map2 would be used when the Redfish client subscribe for the
>> ResourceType to get the Dbus mappings.
>>
>> => Map3 would be used when the Dbus signal gets generated and need
>> the Redfish mappings.
>
> Why can't these be the same mapping?
To start monitoring for Dbus : We need Dbus Resource Path,
Once we get any Dbus signal, we need to send redfish event which needs
redfish resource path hence we need reverse map
Did I get you correctly?
> I think having 3 different maps makes this very confusing. I also
> think this is attempting to generalize the problem too early. If you
> look at the Redfish code to determine some of the more complicated
> properties, sometimes it takes quite a bit of logic. That logic also
> would possibly be nice to reuse. Maybe we can take your idea of a
> match with a callback to some of the already existing property
> parsing? Obviously it would need some cleaning up, but I could see
> something with a map of schemas/properties to function pointers for
> property parsing.
>
>>
>> => Once we have all thsese mapping gets generated and loaded into the
>> memory, bmcweb would start listening
>> for the interfaces listed in map2.
>
> Why aren't these compiled in? I don't see why they need to be loaded,
> could just be in code. Also, they shouldn't be added as matches until
> there is a subscriber, or we'll add many unneeded matches.
They would be compiled in, they would be in memory(big std::map).
Agree monitoring should start only when there is a subscriber.
>
>>
>> => Once any Dbus signal gets generated map3 would be used to get the
>> Redfish mapping.
>>
>> Please let me know if you have any concerns with this approach.
>>
>> Ratan
^ permalink raw reply [flat|nested] 44+ messages in thread
end of thread, other threads:[~2020-07-30 9:10 UTC | newest]
Thread overview: 44+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-31 20:53 Redfish EventService Implementation RAJESWARAN THILLAIGOVINDAN
2020-02-09 20:22 ` RAJESWARAN THILLAIGOVINDAN
2020-02-17 20:11 ` RAJESWARAN THILLAIGOVINDAN
2020-02-19 19:19 ` Puli, Apparao
2020-02-24 6:37 ` Ratan Gupta
2020-02-25 14:06 ` Puli, Apparao
2020-05-05 11:43 ` RAJESWARAN THILLAIGOVINDAN
2020-05-26 12:20 ` RAJESWARAN THILLAIGOVINDAN
2020-05-27 3:48 ` Puli, Apparao
2020-05-27 11:50 ` RAJESWARAN THILLAIGOVINDAN
2020-05-27 18:58 ` Puli, Apparao
2020-05-28 13:26 ` Ratan Gupta
2020-05-29 15:45 ` Redfish event log for new local user addition Puli, Apparao
2020-06-02 6:30 ` Ratan Gupta
2020-06-08 21:08 ` Redfish EventService Implementation Brad Bishop
2020-06-09 0:58 ` James Feist
2020-06-15 12:50 ` Ratan Gupta
2020-06-15 21:42 ` James Feist
2020-06-16 15:24 ` Patrick Williams
2020-06-16 16:11 ` James Feist
2020-06-17 12:08 ` Ratan Gupta
2020-06-17 17:16 ` James Feist
2020-06-17 17:19 ` Bills, Jason M
2020-06-17 18:30 ` Andrew Geissler
2020-06-17 20:45 ` Patrick Williams
2020-06-19 13:26 ` Ratan Gupta
2020-06-19 22:19 ` Bills, Jason M
2020-06-23 7:27 ` Ratan Gupta
2020-06-24 16:24 ` James Feist
2020-06-24 20:39 ` Patrick Williams
2020-06-25 13:45 ` Brad Bishop
2020-06-25 16:42 ` James Feist
2020-06-25 15:49 ` Brad Bishop
2020-06-25 16:44 ` James Feist
2020-06-25 17:26 ` Brad Bishop
2020-06-25 18:55 ` Bills, Jason M
2020-06-29 8:00 ` Ratan Gupta
2020-06-29 8:07 ` Ratan Gupta
2020-06-29 15:33 ` Bills, Jason M
2020-07-03 10:15 ` Ratan Gupta
2020-07-06 21:30 ` Bills, Jason M
2020-07-13 6:32 ` Ratan Gupta
2020-07-14 21:08 ` James Feist
2020-07-30 9:10 ` Ratan Gupta
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.