Re: [RFC 1/2] i3c: Add preliminary support for secondary master

[Przemyslaw Gaj <pgaj@cadence.com>]

Hi Vitor,

The 11/28/2019 15:50, Vitor Soares wrote:
> 
> Hi,
> 
> From: Przemyslaw Gaj <pgaj@cadence.com>
> Date: Thu, Nov 28, 2019 at 12:58:06
> 
> > The 11/28/2019 12:20, Vitor Soares wrote:
> > > 
> > > Hi,
> > > 
> > > 
> > > From: Przemyslaw Gaj <pgaj@cadence.com>
> > > Date: Thu, Nov 28, 2019 at 05:50:08
> > > 
> > > > Hi Vitor,
> > > > 
> > > > First, you woke up my son and he couldn't sleep the rest of the night
> > > > :-)
> > > 
> > > Sorry for that.
> > > 
> > > > I appreciate you sent that so we can discuss it.
> > > > 
> > > > The 11/28/2019 02:15, Vitor Soares wrote:
> > > > > 
> > > > > This patch adds the preliminary support for secondary master feature to
> > > > > i3c Framework for testing purposes.
> > > > > 
> > > > > Key points for consideration:
> > > > >   - mastership_[show/store] are only used for testing
> > > > >   - secondary master registration is made in two steps, one in
> > > > >   i3c_master_register() and another in i3c_sec_master_bus_init() when
> > > > >   secondary master became current master first time. This is made in this
> > > > >   way to get all dt declared boardinfo list, create defslvs list and
> > > > >   provide work_queue.
> > > > >   - When the current master wants to deliver_mastership it necessary to
> > > > >   disable all in-band events to avoid unwanted interrupt during bus
> > > > >   ownership exchange. For now this patch doesn't reflect all
> > > > >   improvements/changes made in v1.1 I3C Bus spec. But it can be extended
> > > > >   by adding some commands and checks to the flow.
> > > > >   - i3c_defslvs_info: The DEFSLVS info can be differently stored in
> > > > >   diferen HC. Hence it is used a defslvs list similar to boardinfo list in
> > > > >   the bus structure to hold this data. Them HC is taccking over the bus
> > > > >   ownership can initialize each device of that list. For now, this not
> > > > >   address the i2c devices since they are only statically described.
> > > > >   - [request/deliver]_mastership(): Mastership request deliver may be done
> > > > >   differently in different HC, hence the need to have a call back for each
> > > > >   process.
> > > > >   - Add dr_mode to DT: Similar to USB, HC can be programmed to Master only
> > > > >   mode, Slave only mode or Secondary Master which aren't necessarily
> > > > >   hardcoded.
> > > > >   - bus_mode definition: The bus mode is defined even without defslvs
> > > > >   information with DT info since the definition of i2c devices are those
> > > > >   that have impact on bus_mode definition and need to statically declared.
> > > > >   The only use case that may cause issues is when i2c devices aren't
> > > > >   declared in secondary master side and bus mode doesn't match the
> > > > >   main master. Anyway this can be solde without extra complexity.
> > > > > 
> > > > > Signed-off-by: Vitor Soares <vitor.soares@synopsys.com>
> > > > > ---
> > > > >  drivers/i3c/master.c       | 365 ++++++++++++++++++++++++++++++++++++++++++++-
> > > > >  include/linux/i3c/master.h |  39 +++++
> > > > >  2 files changed, 396 insertions(+), 8 deletions(-)
> > > > > 
> > > > > diff --git a/drivers/i3c/master.c b/drivers/i3c/master.c
> > > > > index 0436916..b398d77 100644
> > > > > --- a/drivers/i3c/master.c
> > > > > +++ b/drivers/i3c/master.c
> > > > > @@ -449,6 +449,46 @@ static ssize_t mode_show(struct device *dev,
> > > > >  }
> > > > >  static DEVICE_ATTR_RO(mode);
> > > > >  
> > > > > +static ssize_t
> > > > > +mastership_show(struct device *dev, struct device_attribute *da, char *buf)
> > > > > +{
> > > > > +	struct i3c_master_controller *master = dev_to_i3cmaster(dev);
> > > > > +	ssize_t ret;
> > > > > +
> > > > > +	if (master->secondary)
> > > > > +		ret = sprintf(buf, "Secondary Master\n");
> > > > > +	else
> > > > > +		ret = sprintf(buf, "Master\n");
> > > > > +
> > > > > +	return ret;
> > > > > +}
> > > > > +
> > > > > +static ssize_t
> > > > > +mastership_store(struct device *dev, struct device_attribute *attr,
> > > > > +		 const char *buf, size_t count)
> > > > > +{
> > > > > +	struct i3c_master_controller *master = dev_to_i3cmaster(dev);
> > > > > +	struct i3c_bus *i3cbus = dev_to_i3cbus(dev);
> > > > > +
> > > > > +	if (i3cbus->cur_master == master->this) {
> > > > > +		dev_err(dev, "I'm current mater.");
> > > > > +		return count;
> > > > > +	}
> > > > > +
> > > > > +	if (!master->ops->request_mastership) {
> > > > > +		dev_err(dev, "mastership_request not supported.");
> > > > > +		return count;
> > > > > +	}
> > > > > +
> > > > > +	if (master->ops->request_mastership(master))
> > > > > +		dev_err(dev, "mastership_request failed");
> > > > > +	else
> > > > > +		dev_err(dev, "mastership_request success");
> > > > > +
> > > > > +	return count;
> > > > > +}
> > > > > +static DEVICE_ATTR_RW(mastership);
> > > > > +
> > > > >  static ssize_t current_master_show(struct device *dev,
> > > > >  				   struct device_attribute *da,
> > > > >  				   char *buf)
> > > > > @@ -457,8 +497,11 @@ static ssize_t current_master_show(struct device *dev,
> > > > >  	ssize_t ret;
> > > > >  
> > > > >  	i3c_bus_normaluse_lock(i3cbus);
> > > > > -	ret = sprintf(buf, "%d-%llx\n", i3cbus->id,
> > > > > -		      i3cbus->cur_master->info.pid);
> > > > > +	if (i3cbus->cur_master)
> > > > > +		ret = sprintf(buf, "%d-%llx\n", i3cbus->id,
> > > > > +			      i3cbus->cur_master->info.pid);
> > > > > +	else
> > > > > +		ret = sprintf(buf, "Not Current Master\n");
> > > > >  	i3c_bus_normaluse_unlock(i3cbus);
> > > > >  
> > > > >  	return ret;
> > > > > @@ -497,6 +540,7 @@ static DEVICE_ATTR_RO(i2c_scl_frequency);
> > > > >  
> > > > >  static struct attribute *i3c_masterdev_attrs[] = {
> > > > >  	&dev_attr_mode.attr,
> > > > > +	&dev_attr_mastership.attr,
> > > > >  	&dev_attr_current_master.attr,
> > > > >  	&dev_attr_i3c_scl_frequency.attr,
> > > > >  	&dev_attr_i2c_scl_frequency.attr,
> > > > > @@ -854,6 +898,53 @@ int i3c_master_enec_locked(struct i3c_master_controller *master, u8 addr,
> > > > >  EXPORT_SYMBOL_GPL(i3c_master_enec_locked);
> > > > >  
> > > > >  /**
> > > > > + * i3c_master_getaccmst_locked() - send an GETACCMST CCC command
> > > > > + * @master: master used to send frames on the bus
> > > > > + * @addr: a valid I3C slave address
> > > > > + *
> > > > > + * Sends an GETACCMST CCC command to offer bus Mastership to an
> > > > > + * I3C Secondary Master.
> > > > > + *
> > > > > + * This function must be called with the bus lock held in write mode.
> > > > > + *
> > > > > + * Return: 0 in case of success, a positive I3C error code if the error is
> > > > > + * one of the official Mx error codes, and a negative error code otherwise.
> > > > > + */
> > > > > +int i3c_master_getaccmst_locked(struct i3c_master_controller *master, u8 addr)
> > > > > +{
> > > > > +	enum i3c_addr_slot_status addrstat;
> > > > > +	struct i3c_ccc_getaccmst *accmst;
> > > > > +	struct i3c_ccc_cmd_dest dest;
> > > > > +	struct i3c_ccc_cmd cmd;
> > > > > +	int ret;
> > > > > +
> > > > > +	if (!master)
> > > > > +		return -EINVAL;
> > > > > +
> > > > > +	addrstat = i3c_bus_get_addr_slot_status(&master->bus, addr);
> > > > > +	if (addr == I3C_BROADCAST_ADDR || addrstat != I3C_ADDR_SLOT_I3C_DEV)
> > > > > +		return -EINVAL;
> > > > > +
> > > > > +	accmst = i3c_ccc_cmd_dest_init(&dest, addr, sizeof(*accmst));
> > > > > +	if (!accmst)
> > > > > +		return -ENOMEM;
> > > > > +
> > > > > +	i3c_ccc_cmd_init(&cmd, true, I3C_CCC_GETACCMST, &dest, 1);
> > > > > +
> > > > > +	ret = i3c_master_send_ccc_cmd_locked(master, &cmd);
> > > > > +	if (ret)
> > > > > +		goto out;
> > > > > +
> > > > > +	if (accmst->newmaster >> 1 != addr)
> > > > 
> > > > I really like this check. This is something I realized working
> > > > on next patch version.
> > > > 
> > > > > +		ret = -EIO;
> > > > > +out:
> > > > > +	i3c_ccc_cmd_dest_cleanup(&dest);
> > > > > +
> > > > > +	return ret;
> > > > > +}
> > > > > +EXPORT_SYMBOL_GPL(i3c_master_getaccmst_locked);
> > > > > +
> > > > > +/**
> > > > >   * i3c_master_defslvs_locked() - send a DEFSLVS CCC command
> > > > >   * @master: master used to send frames on the bus
> > > > >   *
> > > > > @@ -1542,8 +1633,7 @@ int i3c_master_set_info(struct i3c_master_controller *master,
> > > > >  	if (!i3c_bus_dev_addr_is_avail(&master->bus, info->dyn_addr))
> > > > >  		return -EINVAL;
> > > > >  
> > > > > -	if (I3C_BCR_DEVICE_ROLE(info->bcr) == I3C_BCR_I3C_MASTER &&
> > > > > -	    master->secondary)
> > > > > +	if (I3C_BCR_DEVICE_ROLE(info->bcr) != I3C_BCR_I3C_MASTER)
> > > > >  		return -EINVAL;
> > > > >  
> > > > >  	if (master->this)
> > > > > @@ -2381,6 +2471,81 @@ static int i3c_master_check_ops(const struct i3c_master_controller_ops *ops)
> > > > >  	return 0;
> > > > >  }
> > > > >  
> > > > > +int i3c_sec_master_bus_init(struct i3c_master_controller *master)
> > > > > +{
> > > > > +	unsigned long i2c_scl_rate = I3C_BUS_I2C_FM_PLUS_SCL_RATE;
> > > > > +	struct i3c_bus *i3cbus = i3c_master_get_bus(master);
> > > > > +	enum i3c_bus_mode mode = i3cbus->mode;
> > > > > +	struct i3c_defslvs_info *defslvsinfo;
> > > > > +	int ret = 0;
> > > > > +
> > > > > +	if (master->init_done)
> > > > > +		return -EINVAL;
> > > > > +
> > > > > +	list_for_each_entry(defslvsinfo, &master->defslvs, node) {
> > > > > +		if (defslvsinfo->slave.dyn_addr)
> > > > > +			continue;
> > > > > +
> > > > > +		switch (defslvsinfo->slave.lvr & I3C_LVR_I2C_INDEX_MASK) {
> > > > > +		case I3C_LVR_I2C_INDEX(0):
> > > > > +			if (mode < I3C_BUS_MODE_MIXED_FAST)
> > > > > +				mode = I3C_BUS_MODE_MIXED_FAST;
> > > > > +			break;
> > > > > +		case I3C_LVR_I2C_INDEX(1):
> > > > > +		case I3C_LVR_I2C_INDEX(2):
> > > > > +			if (mode < I3C_BUS_MODE_MIXED_SLOW)
> > > > > +				mode = I3C_BUS_MODE_MIXED_SLOW;
> > > > > +			break;
> > > > > +		default:
> > > > > +			ret = -EINVAL;
> > > > > +			goto err_put_dev;
> > > > > +		}
> > > > > +		if (defslvsinfo->slave.lvr & I3C_LVR_I2C_FM_MODE)
> > > > > +			i2c_scl_rate = I3C_BUS_I2C_FM_SCL_RATE;
> > > > > +	}
> > > > > +
> > > > > +	ret = i3c_bus_set_mode(i3cbus, mode, i2c_scl_rate);
> > > > > +	if (ret)
> > > > > +		goto err_put_dev;
> > > > > +
> > > > > +	/*
> > > > > +	 * Now execute the controller specific ->bus_init() routine, which
> > > > > +	 * might configure its internal logic to match the bus limitations.
> > > > > +	 */
> > > > > +	ret = master->ops->bus_init(master);
> > > > > +	if (ret)
> > > > > +		goto err_put_dev;
> > > > > +
> > > > > +	/*
> > > > > +	 * The master device should have been instantiated in ->bus_init(),
> > > > > +	 * complain if this was not the case.
> > > > > +	 */
> > > > > +	if (!master->this) {
> > > > > +		dev_err(&master->dev,
> > > > > +			"master_set_info() was not called in ->bus_init()\n");
> > > > > +		ret = -EINVAL;
> > > > > +		goto err_put_dev;
> > > > > +	}
> > > > > +
> > > > > +	ret = device_add(&master->dev);
> > > > > +	if (ret)
> > > > > +		return ret;
> > > > > +
> > > > > +	/*
> > > > > +	 * Expose our I3C bus as an I2C adapter so that I2C devices are exposed
> > > > > +	 * through the I2C subsystem.
> > > > > +	 */
> > > > > +	ret = i3c_master_i2c_adapter_init(master);
> > > > > +	if (ret)
> > > > > +		goto err_put_dev;
> > > > > +
> > > > > +	master->init_done = true;
> > > > > +
> > > > > +err_put_dev:
> > > > > +	return ret;
> > > > > +}
> > > > > +EXPORT_SYMBOL_GPL(i3c_sec_master_bus_init);
> > > > > +
> > > > >  /**
> > > > >   * i3c_master_register() - register an I3C master
> > > > >   * @master: master used to send frames on the bus
> > > > > @@ -2413,10 +2578,6 @@ int i3c_master_register(struct i3c_master_controller *master,
> > > > >  	struct i2c_dev_boardinfo *i2cbi;
> > > > >  	int ret;
> > > > >  
> > > > > -	/* We do not support secondary masters yet. */
> > > > > -	if (secondary)
> > > > > -		return -ENOTSUPP;
> > > > > -
> > > > >  	ret = i3c_master_check_ops(ops);
> > > > >  	if (ret)
> > > > >  		return ret;
> > > > > @@ -2430,6 +2591,7 @@ int i3c_master_register(struct i3c_master_controller *master,
> > > > >  	master->secondary = secondary;
> > > > >  	INIT_LIST_HEAD(&master->boardinfo.i2c);
> > > > >  	INIT_LIST_HEAD(&master->boardinfo.i3c);
> > > > > +	INIT_LIST_HEAD(&master->defslvs);
> > > > >  
> > > > >  	ret = i3c_bus_init(i3cbus);
> > > > >  	if (ret)
> > > > > @@ -2475,6 +2637,9 @@ int i3c_master_register(struct i3c_master_controller *master,
> > > > >  		goto err_put_dev;
> > > > >  	}
> > > > >  
> > > > > +	if (secondary)
> > > > > +		return 0;
> > > > > +
> > > > >  	ret = i3c_master_bus_init(master);
> > > > >  	if (ret)
> > > > >  		goto err_put_dev;
> > > > > @@ -2547,6 +2712,11 @@ int i3c_dev_do_priv_xfers_locked(struct i3c_dev_desc *dev,
> > > > >  	if (!master || !xfers)
> > > > >  		return -EINVAL;
> > > > >  
> > > > > +	if (master->bus.cur_master != master->this) {
> > > > > +		master->ops->request_mastership(master);
> > > > > +		return -EBUSY;
> > > > 
> > > > I don't like this approach, so you have to re-trigger the operation when
> > > > this master becomes current master. It is not transparent. Especially,
> > > > HCI 1.1 describes the process in detail, even  on flow chart and you can 
> > > > see there that you should block all the transfers/tasks on your side and
> > > > wait for GETACCMST.
> > > 
> > > I forgot to explain that this is part is not fully operational and my 
> > > intention was to address this in near future.
> > > To quickly head-up, what I had in my mind when did this was if a device 
> > > wants to do a xfer and sec master is not current master the framework 
> > > will request the bus ownership and them pass a EBUSY in case of success 
> > > or another error in case current master Nacks the MR request.
> > > 
> > > > 
> > > > > +	}
> > > > > +
> > > > >  	if (!master->ops->priv_xfers)
> > > > >  		return -ENOTSUPP;
> > > > >  
> > > > > @@ -2638,6 +2808,185 @@ void i3c_dev_free_ibi_locked(struct i3c_dev_desc *dev)
> > > > >  	dev->ibi = NULL;
> > > > >  }
> > > > >  
> > > > > +static const char *const i3c_dr_modes[] = {
> > > > > +	[I3C_DR_MODE_MASTER]		= "master",
> > > > > +	[I3C_DR_MODE_SEC_MASTER]	= "sec-master",
> > > > > +	[I3C_DR_MODE_SLAVE]		= "slave",
> > > > > +};
> > > > > +
> > > > > +static enum i3c_dr_mode i3c_get_dr_mode_from_string(const char *str)
> > > > > +{
> > > > > +	int ret;
> > > > > +
> > > > > +	ret = match_string(i3c_dr_modes, ARRAY_SIZE(i3c_dr_modes), str);
> > > > > +	return (ret < 0) ? I3C_DR_MODE_MASTER : ret;
> > > > > +}
> > > > > +
> > > > > +enum i3c_dr_mode i3c_get_dr_mode(struct device *dev)
> > > > > +{
> > > > > +	const char *dr_mode;
> > > > > +	int err;
> > > > > +
> > > > > +	err = device_property_read_string(dev, "dr-mode", &dr_mode);
> > > > > +	if (err < 0)
> > > > > +		return I3C_DR_MODE_MASTER;
> > > > > +
> > > > > +	return i3c_get_dr_mode_from_string(dr_mode);
> > > > > +}
> > > > > +EXPORT_SYMBOL_GPL(i3c_get_dr_mode);
> > > > > +
> > > > > +int i3c_sec_master_request_mastership(struct i3c_master_controller *master)
> > > > > +{
> > > > > +	int ret;
> > > > > +
> > > > > +	i3c_bus_normaluse_lock(&master->bus);
> > > > > +	ret = master->ops->request_mastership(master);
> > > > > +	i3c_bus_normaluse_unlock(&master->bus);
> > > > > +
> > > > > +	return ret;
> > > > > +}
> > > > > +EXPORT_SYMBOL_GPL(i3c_sec_master_request_mastership);
> > > > > +
> > > > > +int i3c_master_deliver_mastership(struct i3c_master_controller *master, u8 addr)
> > > > 
> > > > I agree, wa can introduce this now. But we decided to postpone it. As
> > > > you can see, it shouldn't be so hard.
> > > 
> > > IMO this is important, I could use directly the CCC but I know that there 
> > > is other HC that may use a different approach.
> > > 
> > > > 
> > > > > +{
> > > > > +	struct i3c_dev_desc *dev;
> > > > > +	int ret;
> > > > > +
> > > > > +	i3c_bus_normaluse_lock(&master->bus);
> > > > > +	i3c_bus_for_each_i3cdev(&master->bus, dev) {
> > > > > +		if (dev->ibi) {
> > > > > +			mutex_lock(&dev->ibi_lock);
> > > > > +			i3c_dev_disable_ibi_locked(dev);
> > > > > +			mutex_unlock(&dev->ibi_lock);
> > > > > +		}
> > > > > +	}
> > > > > +	i3c_master_disec_locked(master, I3C_BROADCAST_ADDR,
> > > > > +				I3C_CCC_EVENT_MR | I3C_CCC_EVENT_HJ);
> > > > > +
> > > > > +	ret = master->ops->deliver_mastership(master, addr);
> > > > > +	i3c_bus_normaluse_unlock(&master->bus);
> > > > > +
> > > > > +	return ret;
> > > > > +}
> > > > > +EXPORT_SYMBOL_GPL(i3c_master_deliver_mastership);
> > > > > +
> > > > > +int defslvs_populate_i3c_bus(struct i3c_master_controller *master,
> > > > > +			     struct i3c_ccc_dev_desc defslvs)
> > > > > +{
> > > > > +	struct i3c_defslvs_info *defslvsinfo;
> > > > > +	struct device *dev = &master->dev;
> > > > > +
> > > > > +	i3c_bus_maintenance_lock(&master->bus);
> > > > > +	defslvsinfo = devm_kzalloc(dev, sizeof(*defslvsinfo), GFP_KERNEL);
> > > > > +	if (!defslvsinfo)
> > > > > +		return -ENOMEM;
> > > > > +
> > > > > +	defslvsinfo->slave = defslvs;
> > > > > +
> > > > > +	list_add_tail(&defslvsinfo->node, &master->defslvs);
> > > > 
> > > > I don't get why can't we call i3c_master_add_i3c_dev_locked when
> > > > populating the bus.
> > > 
> > > I want to avoid the populating bus call back.
> > 
> > Why?
> 
> When I developed this procedure the idea behind it was to avoid bus 
> callbacks and offload this complexity to framework because we don't know 
> how others HC handle DEFSLVS CCC, lets image the case where the DEFSLVS 
> are stored in a place that can be rewriter  before sec master get bus 
> ownership.

Yes, it is theoretically possible. This is not the case for our
controller, HCI and as I can see for DW also not. As now we have only
few drivers, I think we can do it this way.

In HCI, this data is stored in DAT table and entries has to be populated
because index in DAT table is used for transfer descriptors. Without
that, HC is not able to identify target device. Again, we should be ok
with it. The key here is to make the API as straightforward as possible

> Another advantage that I see is, when someone write a new HC driver won't 
> need to care about this because the framework already provide this 
> facility.

We have 2 drivers for now, I think that those two + HCI should be
satisfied now.

> 
> > 
> > > 
> > > > You have all the data on your plate (in HC driver)
> > > > when you are populating it from SEC_DEV_CHAR_TABLE_LOC.
> > > 
> > > Yes, I told you that I have a table for that, yet I decide to not use it. 
> > > My concern is about the HC that doesn't have?
> > > For me passing this task for the framework is more universal. 
> > > 
> > 
> > Looks like we can do the same in HCI.
> > HC driver has to know always the devices on the bus.
> 
> My view here is more like the HC driver provide a way of the framework 
> interact with bus.
> 

API extends that way, and we are dupplicating the code. If we decide at
some point that we should provide information to framewoek this way, we
can change it easily.

> > 
> > > > 
> > > > I decided to do it similarly, but then Boris suggested to rework it and
> > > > we use only i3c_master_add_i3c_dev_locked.
> > > > 
> > > > > +
> > > > > +	i3c_bus_maintenance_unlock(&master->bus);
> > > > > +
> > > > > +	return 0;
> > > > > +}
> > > > > +EXPORT_SYMBOL_GPL(defslvs_populate_i3c_bus);
> > > > > +
> > > > > +static void i3c_master_add_new_defslvs(struct i3c_master_controller *master)
> > > > > +{
> > > > > +	struct i3c_defslvs_info *defslvsinfo;
> > > > > +
> > > > > +	list_for_each_entry(defslvsinfo, &master->defslvs, node) {
> > > > > +		/* TODO: add i2c devices to the bus */
> > > > > +		if (!defslvsinfo->slave.dyn_addr)
> > > > > +			continue;
> > > > > +
> > > > > +		if (defslvsinfo->slave.dyn_addr == master->this->info.dyn_addr)
> > > > > +			continue;
> > > > > +
> > > > > +		if (!i3c_bus_dev_addr_is_avail(&master->bus,
> > > > > +					       defslvsinfo->slave.dyn_addr))
> > > > 
> > > > We can add those checks also but we also have i3c_master_attach_i3c_dev
> > > > and i3c_master_get_i3c_addrs which takes care of this everything.
> > > 
> > > Yes, but them you will be allocating and free devs unnecessarily.
> > > 
> > 
> > Can you explain that?
> 
> Yap. In i3c_master_add_i3c_dev_locked we 1st allocate the dev, then 
> attach the device and if it fails we free the dev.
> I think here we can check in upfront if the address is already in use and 
> avoid extra steps.

Yes, I agree. We can do that. We cand sent this patch earlier or later
to not to complicate mastership patchset. I want to keep it simple.

> 
> > 
> > > > 
> > > > > +			continue;
> > > > > +
> > > > > +		i3c_master_add_i3c_dev_locked(master, defslvsinfo->slave.dyn_addr);
> > > > > +	}
> > > > > +
> > > > > +	while (!list_empty(&master->defslvs)) {
> > > > > +		defslvsinfo = list_first_entry(&master->defslvs,
> > > > > +					       struct i3c_defslvs_info, node);
> > > > > +		list_del(&defslvsinfo->node);
> > > > 
> > > > I feel like this code is redundant, we have to allocate it, then delete.
> > > 
> > > No. you need to clean the list. I may receive another one in the future 
> > > due a HJ or a dynamic address change.
> > >
> 
> In my previous comment I forgot to mention by doing the cleanup in API we 
> avoid duplicated code in HC drivers.

When you call i3c_master_add_i3d_dev_locked directly, code is not
dupplicated. We can rework also device registration on HC side to not to
introduce dupplicates.

> 
> > 
> > I like more the way when HC driver is updateing that. It makes API
> > simpler.
> 
> There is nothing against that 😉. In the end, the purpose of this DEFSLVS 
> handling was to present an alternative solution and its advantages.

Maybe someday we'll be forced to introduce that, who knows. For now,
for everyone, key should be to keep it simple.

> 
> > 
> > > > 
> > > > > +	}
> > > > > +}
> > > > > +
> > > > > +int i3c_master_switch_operation_mode(struct i3c_master_controller *master,
> > > > > +				     bool secondary)
> > > > > +{
> > > > > +	struct i3c_dev_desc *dev;
> > > > > +	int ret;
> > > > > +
> > > > > +	if (master->secondary == secondary)
> > > > > +		return -EEXIST;
> > > > > +
> > > > > +	/* TODO: get the current master information */
> > > > > +	if (secondary)
> > > > > +		master->bus.cur_master = NULL;
> > > > > +	else
> > > > > +		master->bus.cur_master = master->this;
> > > > > +
> > > > > +	if (!master->init_done && !secondary)
> > > > > +		i3c_sec_master_bus_init(master);
> > > > > +
> > > > > +	master->secondary = secondary;
> > > > > +
> > > > > +	dev_info(&master->dev, "changing to %s\n",
> > > > > +		 master->secondary ? "Sec Master" : "Master");
> > > > > +
> > > > > +	/* TODO: Analyse the use of maintenan_lock for everything */
> > > > > +	if (!list_empty(&master->defslvs) && !secondary) {
> > > > > +		i3c_bus_maintenance_lock(&master->bus);
> > > > > +		i3c_master_add_new_defslvs(master);
> > > > > +		i3c_bus_maintenance_unlock(&master->bus);
> > > > > +
> > > > > +		i3c_bus_normaluse_lock(&master->bus);
> > > > > +		i3c_master_register_new_i3c_devs(master);
> > > > 
> > > > Take a look also at i3c_master_bus_takeover from my latest patch. BTW.
> > > > what about I2C devices? We worked on that also, and this is part of the
> > > > latest patch also. I'm testing it with I2C devices also.
> > > 
> > > Please check the comments for that. Anyway you can declare the i2c 
> > > devices of DT on both sides, what I think it should be a good practice.
> > > 
> > 
> > This is what we are doing, but we are comparing I2C devices from DEFSLVS
> > list with those defined in DT. We discussed that already.
> 
> You are right.
> 
> > 
> > > > 
> > > > > +		i3c_bus_normaluse_unlock(&master->bus);
> > > > > +	}
> > > > > +
> > > > > +	if (!secondary) {
> > > > > +		i3c_bus_normaluse_lock(&master->bus);
> > > > > +		i3c_bus_for_each_i3cdev(&master->bus, dev) {
> > > > > +			if (dev->ibi) {
> > > > > +				mutex_lock(&dev->ibi_lock);
> > > > > +				ret = i3c_dev_enable_ibi_locked(dev);
> > > > > +				if (ret)
> > > > > +					dev_err(&master->dev,
> > > > > +						"Failed to re-enable IBI on device %d-%llx",
> > > > > +						master->bus.id, dev->info.pid);
> > > > > +				mutex_unlock(&dev->ibi_lock);
> > > > > +				}
> > > > > +		}
> > > > > +
> > > > > +		/* TODO: Enable MR only for the elegible devices */
> > > > 
> > > > This was postponed also, but we had that before. We can add per-device
> > > > granularity to i3c_master_bus_takeover().
> > > 
> > > Here we need to do the same as for ibi and maybe get the eligible devices 
> > > from DT or based on its DCR,BCR and PID.
> > > This is something that should be address because it represent a big 
> > > security gap.
> > > 
> > > > 
> > > > > +		i3c_master_enec_locked(master, I3C_BROADCAST_ADDR,
> > > > > +					I3C_CCC_EVENT_MR | I3C_CCC_EVENT_HJ);
> > > > > +		i3c_bus_normaluse_unlock(&master->bus);
> > > > > +	}
> > > > > +
> > > > > +	return 0;
> > > > > +}
> > > > > +EXPORT_SYMBOL_GPL(i3c_master_switch_operation_mode);
> > > > > +
> > > > > +int i3c_for_each_dev(void *data, int (*fn)(struct device *, void *))
> > > > > +{
> > > > > +	int res;
> > > > > +
> > > > > +	mutex_lock(&i3c_core_lock);
> > > > > +	res = bus_for_each_dev(&i3c_bus_type, NULL, data, fn);
> > > > > +	mutex_unlock(&i3c_core_lock);
> > > > > +
> > > > > +	return res;
> > > > > +}
> > > > > +EXPORT_SYMBOL_GPL(i3c_for_each_dev);
> > > > > +
> > > > >  static int __init i3c_init(void)
> > > > >  {
> > > > >  	return bus_register(&i3c_bus_type);
> > > > > diff --git a/include/linux/i3c/master.h b/include/linux/i3c/master.h
> > > > > index 9cb39d9..09bd99c 100644
> > > > > --- a/include/linux/i3c/master.h
> > > > > +++ b/include/linux/i3c/master.h
> > > > > @@ -426,6 +426,8 @@ struct i3c_bus {
> > > > >   *		      for a future IBI
> > > > >   *		      This method is mandatory only if ->request_ibi is not
> > > > >   *		      NULL.
> > > > > + * @request_mastership: Request mastership.
> > > > > + * @deliver_mastership: Deliver mastership
> > > > >   */
> > > > >  struct i3c_master_controller_ops {
> > > > >  	int (*bus_init)(struct i3c_master_controller *master);
> > > > > @@ -452,6 +454,21 @@ struct i3c_master_controller_ops {
> > > > >  	int (*disable_ibi)(struct i3c_dev_desc *dev);
> > > > >  	void (*recycle_ibi_slot)(struct i3c_dev_desc *dev,
> > > > >  				 struct i3c_ibi_slot *slot);
> > > > > +	int (*request_mastership)(struct i3c_master_controller *master);
> > > > > +	int (*deliver_mastership)(struct i3c_master_controller *master,
> > > > > +				  u8 addr);
> > > > > +};
> > > > > +
> > > > > +/**
> > > > > + * struct i3c_defslvs_info - defslvs information object
> > > > > + * @node: used to insert the defslvs object in the  list
> > > > > + * @slave: I3C/I2C device descriptor used for DEFSLVS
> > > > > + *
> > > > > + * This structure is used to hold defslvs information on Secondary Master.
> > > > > + */
> > > > > +struct i3c_defslvs_info {
> > > > > +	struct list_head node;
> > > > > +	struct i3c_ccc_dev_desc slave;
> > > > >  };
> > > > >  
> > > > >  /**
> > > > > @@ -468,6 +485,7 @@ struct i3c_master_controller_ops {
> > > > >   * @boardinfo.i3c: list of I3C  boardinfo objects
> > > > >   * @boardinfo.i2c: list of I2C boardinfo objects
> > > > >   * @boardinfo: board-level information attached to devices connected on the bus
> > > > > + * @defslvs: List of defslvs objects
> > > > >   * @bus: I3C bus exposed by this master
> > > > >   * @wq: workqueue used to execute IBI handlers. Can also be used by master
> > > > >   *	drivers if they need to postpone operations that need to take place
> > > > > @@ -491,6 +509,7 @@ struct i3c_master_controller {
> > > > >  		struct list_head i3c;
> > > > >  		struct list_head i2c;
> > > > >  	} boardinfo;
> > > > > +	struct list_head defslvs;
> > > > >  	struct i3c_bus bus;
> > > > >  	struct workqueue_struct *wq;
> > > > >  };
> > > > > @@ -525,6 +544,7 @@ int i3c_master_disec_locked(struct i3c_master_controller *master, u8 addr,
> > > > >  			    u8 evts);
> > > > >  int i3c_master_enec_locked(struct i3c_master_controller *master, u8 addr,
> > > > >  			   u8 evts);
> > > > > +int i3c_master_getaccmst_locked(struct i3c_master_controller *master, u8 addr);
> > > > >  int i3c_master_entdaa_locked(struct i3c_master_controller *master);
> > > > >  int i3c_master_defslvs_locked(struct i3c_master_controller *master);
> > > > >  
> > > > > @@ -652,4 +672,23 @@ void i3c_master_queue_ibi(struct i3c_dev_desc *dev, struct i3c_ibi_slot *slot);
> > > > >  
> > > > >  struct i3c_ibi_slot *i3c_master_get_free_ibi_slot(struct i3c_dev_desc *dev);
> > > > >  
> > > > > +enum i3c_dr_mode {
> > > > > +	I3C_DR_MODE_MASTER,
> > > > > +	I3C_DR_MODE_SEC_MASTER,
> > > > > +	I3C_DR_MODE_SLAVE,
> > > > > +};
> > > > > +
> > > > > +enum i3c_dr_mode i3c_get_dr_mode(struct device *dev);
> > > > > +
> > > > > +int i3c_master_switch_operation_mode(struct i3c_master_controller *master,
> > > > > +				     bool secondary);
> > > > > +
> > > > > +int i3c_sec_master_request_mastership(struct i3c_master_controller *master);
> > > > > +int i3c_master_deliver_mastership(struct i3c_master_controller *master,
> > > > > +				  u8 addr);
> > > > > +
> > > > > +int i3c_sec_master_bus_init(struct i3c_master_controller *master);
> > > > > +int defslvs_populate_i3c_bus(struct i3c_master_controller *master,
> > > > > +			     struct i3c_ccc_dev_desc defslvs);
> > > > > +
> > > > >  #endif /* I3C_MASTER_H */
> > > > > -- 
> > > > > 2.7.4
> > > > > 
> > > > 
> > > > I feel like this is based on early approach which has evolved by the time
> > > > and I think some of the nice part are missing. The biggest parts to
> > > > discuass are:
> > > > - bus population using devslvs device list instead of using
> > > >   i3c_master_add_i3c_dev locked, have you tried that? If something does
> > > >   not work for you, wa can adjust it but I really want you to try the
> > > >   new approach
> > > 
> > > To me the i3c_master_add_i3c_dev_locked just need an improvement to 
> > > propagate the boardinfo to all devices, something that already I started, 
> > > and we need to discuss if on secondary master side we can change the 
> > > devices addresses which imply to send DEFSLVS at the end of the process.
> > 
> > So let's improve it. Can you point me to that?
> 
> Regarding the improvements you can check the patches that touch in the 
> API in this series [1].
> During the development of v3 I started to thinking it is a good idea to 
> pre-reserve all declared addresses in DT and not use them during the 
> ENTDAA process. We can discuss this when I send the new version.
> 

Ok, I'll look into that once again implementing the new version. But I
still feel like we can detach that but we should keep the information
about the device in the framework. Otherwise we are losing some
information which may be useful in the future.

> 
> Regarding the secondary master, do you think it is a good idea to let the 
> secondary master change the dynamic address of a device assigned by the 
> main master?
> For me we might need this kind of algorithms thing to define the 
> secondary master behavior.
> 

The I3C spec clearly defines that case. Secondary master can reconfigure
the bus. But, depending on PID we are looking for dupplicates and
reattaching the devices with new DAs. I'm testing that part also,
introduced that test after our last discussion. Of course, I'm forcing
it currently from secondary master side but this is the proof :-)

> > 
> > > 
> > > > 
> > > > - the way how we are requesting mastership, IMO, we should wait untill
> > > >   the process is finished, as also described in HCI spec for example.
> > > 
> > > I think you are confusing the concept here. I understand you want to do 
> > > the xfer straight way after receive GETACCMST and I agree with that.
> > > The thing is that you don't know what will happen on the bus between the 
> > > time you send the request and you get the bus ownership. I just think we 
> > > need to find another solution for that like defer the transfer to another 
> > > time and when the controller switch the rule trigger all transfers in the 
> > > pipeline (something like this) even before restore all ibi. If you block 
> > > the bus you are not able to pass the defslvs to the framework.  
> > > 
> > 
> > If the MR request was acked, we know that we'll get the control. And it
> > doesn't matter if current master will enable/disable something, change
> > the activity state, send another DEFSLVS, we still want to make a
> > transfer. 
> > 
> > In your approach it is possible that we get the control, we'll
> > not make the transfer and give control back to a diffent master. 
> > I have tests for more than 2 masters.
> 
> You are right.
> This case I didn't address it yet. I have a upper layer that give bus 
> access time to each master for now.
> 
> In your use case if you don't need the IBI in the secondary master side, 
> I would say the best is to request mastership, do the transfer, and send 
> back the mastership, without the logistics enable/disable IBI in 
> secondary master.
> 
> > 
> > And also, I don't feel like queueing the transfers is good idea. When
> > some upper layer wants to transfer the data again, old data may be stale
> > but we still will transfer that.
> 
> That might be a problem for both approach because the exchange of 
> ownership can take so long that xfer data might not be valid anymore.

But we shouldn't wait forever. And this should give upper layer the
control, if mastership is not possible, upper layer may just retry the
transfer later, if still needed. Also, MIPI spec describes this: "After
acknowledging the Mastership Request, the Current Master should prepare
for Handoff in a timely manner".

> IMO it is also dangerous to have the timeout of the bus ownership 
> exchange in HC driver because it will depend from use case to use case 
> and it is not something predictable.

Exactly, that's why I'm going to do one thing, I discussed that already
with Boris. I'll add timeout to the HC driver struct for now, and then
we can change it and pass that timeout to the function. This wy we can
potentialy expose that timeout through sysfs to let users tweak it
depending on their use case.

> 
> > 
> > > >   When I introduce interrupt-based solution, everything should be fine.
> > > >   Could you do that also in your driver?
> > > 
> > > I did it but had issues and didn't fit in my use case. That the reason I 
> > > did this approach.
> > 
> > Can you describe what issues you faced with this approach? What is your
> > use case? If we get the control, we just make the transfers, if not,
> > return exit code to upper layer and that's it.
> 
> The way I have the code that is not possible. In fact it is something 
> that I need help to let the xfer continue after sec master takeover the 
> bus. 

I think this approach makes it too complex. User can handle the cases
when MR wasn't possible. And the transfers will be handled
automatically.

> 
> > 
> > > 
> > > > 
> > > > Again, good to have that. I'm really counting on a fair discussion :-)
> > > > 
> > > > -- 
> > > > --
> > > > Regards, 
> > > > Przemyslaw Gaj
> > > 
> > > As you can see this is based on your work. Off course it as some lose 
> > > ends that should be addressed (like i2c devices, bus mode...) but don't 
> > > have big impact in what I want to show.
> > 
> > Yes, but on my work in early stage and I think this approach wasn't
> > perfect. Then we decided to refork some parts which is not part of that.
> > I feel like we can't find common view.
> > 
> > > For me it is important to rely on framework the way how bus ownership 
> > > exchange is made because it will be easier to maintain in long term and 
> > > we can introduce algorithms for the bus access in the future.
> > > 
> > 
> > Yes, why not. My latest patchset does not break anything here.
> 
> It already passed few months since them, but I remember to have issues 
> during the master/secondary registration.
> 

Ok, I'll take few things from here to next release but the question is,
can you take it after that and try to implement MR in DW based on my
work? It will really help, I hope I can count on you :-). Without that
we won't be able to find the solution.

> > 
> > > If you are using i3c-tree/next you can apply this directly and if you 
> > > find anything else that isn't going ok please let me know.
> > > 
> > > Best regards,
> > > Vitor Soares
> > 
> > -- 
> > -- 
> > Regards,
> > Przemyslaw Gaj
> 
> Best regards,
> Vitor Soares
> 
> [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__patchwork.kernel.org_patch_11127661_&d=DwIGaQ&c=aUq983L2pue2FqKFoP6PGHMJQyoJ7kl3s3GZ-_haXqY&r=CMnAfM_OfpqcWZRfiqcRWw&m=PaFCjYnrERfxLlGyJ94Fp6dSCEEIT9dyKDOOHqenQW0&s=Nu8sd9WAbwad39vbwVYSc3PN5TmLRnlya2l-vujYFHo&e= 
> 

-- 
-- 
Regards,
Przemek

_______________________________________________
linux-i3c mailing list
linux-i3c@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-i3c