Re: [meta-raspberrypi] Network not working after first boot success

Re: [meta-raspberrypi] Network not working after first boot success

[Rich Bayliss]

I am building rpi-basic-image and I get a successful build and
first-run. My Pi gets an address over DHCP and I can login over SSH.

However, if I pull the power/reboot - then on the next start-up my
network stays down. On the local console, if I issue "ifup eth0" I am
told the interface is already up, while "ifconfig" show no interfaces,
not even "lo". If I do and "ifdown eth0" followed by an "ifup eth0"
then my DHCP kicks in and everything is back to normal... until the
next reboot.

Is this a known issue? Does anyone have any ideas?

-- 
Rich Bayliss

Re: [meta-raspberrypi] Network not working after first boot success

[Andrei Gherzan]

[-- Attachment #1: Type: text/plain, Size: 742 bytes --]

On Wed, Jul 17, 2013 at 11:12 PM, Rich Bayliss <richbayliss@gmail.com>wrote:

> I am building rpi-basic-image and I get a successful build and
> first-run. My Pi gets an address over DHCP and I can login over SSH.
>
> However, if I pull the power/reboot - then on the next start-up my
> network stays down. On the local console, if I issue "ifup eth0" I am
> told the interface is already up, while "ifconfig" show no interfaces,
> not even "lo". If I do and "ifdown eth0" followed by an "ifup eth0"
> then my DHCP kicks in and everything is back to normal... until the
> next reboot.
>
> Is this a known issue? Does anyone have any ideas?
>
>
> Never seen this but will try to reproduce it.
Will provide feedback.

Andrei

[-- Attachment #2: Type: text/html, Size: 1249 bytes --]

Re: [meta-raspberrypi] Network not working after first boot success

[Rich Bayliss]

On 18 July 2013 09:01, Andrei Gherzan <andrei@gherzan.ro> wrote:
>
>
>
> On Wed, Jul 17, 2013 at 11:12 PM, Rich Bayliss <richbayliss@gmail.com>
> wrote:
>>
>> I am building rpi-basic-image and I get a successful build and
>> first-run. My Pi gets an address over DHCP and I can login over SSH.
>>
>> However, if I pull the power/reboot - then on the next start-up my
>> network stays down. On the local console, if I issue "ifup eth0" I am
>> told the interface is already up, while "ifconfig" show no interfaces,
>> not even "lo". If I do and "ifdown eth0" followed by an "ifup eth0"
>> then my DHCP kicks in and everything is back to normal... until the
>> next reboot.
>>
>> Is this a known issue? Does anyone have any ideas?
>>
>>
> Never seen this but will try to reproduce it.
> Will provide feedback.
>
> Andrei

Thanks Andrei, I would appreciate that.

For reference, I have downloaded the latest Raspbian image and I have
no such issue - so I would consider this as ruling out a hardware
issue.

Also, I have noticed the Ethernet activity LEDs go off and on, during
boot, when the interface is started correctly. When I have the issue,
they come on initially but never go off/on again. Maybe an
initialization error?

--
Rich Bayliss

Re: [meta-raspberrypi] Network not working after first boot success

[Paul Barker]

On 18 July 2013 11:55, Rich Bayliss <richbayliss@gmail.com> wrote:
>>>
>>> Is this a known issue? Does anyone have any ideas?
>>>

Could you try running 'ip addr' and 'ip link' instead of 'ifconfig'?
Just wondering whether they query information differently and might
show something else. The other place to look is in the output of
'dmesg'.

Does un-plugging and re-plugging the network cable (without running
'ifup' or 'ifdown') make any difference?

--
Paul Barker

Email: paul@paulbarker.me.uk
http://www.paulbarker.me.uk

Re: [meta-raspberrypi] Network not working after first boot success

[Rich Bayliss]

On 18 July 2013 12:07, Paul Barker <paul@paulbarker.me.uk> wrote:
> On 18 July 2013 11:55, Rich Bayliss <richbayliss@gmail.com> wrote:
>>>>
>>>> Is this a known issue? Does anyone have any ideas?
>>>>
>
> Could you try running 'ip addr' and 'ip link' instead of 'ifconfig'?
> Just wondering whether they query information differently and might
> show something else. The other place to look is in the output of
> 'dmesg'.
>
> Does un-plugging and re-plugging the network cable (without running
> 'ifup' or 'ifdown') make any difference?
>
> --
> Paul Barker
>
> Email: paul@paulbarker.me.uk
> http://www.paulbarker.me.uk

I have tried unplugging/replugging and it stays down.

I have run the commands and there is a difference, however I also ran
"lsmod" and there is a big difference.

Working:
root@raspberrypi:~# lsmod
    Not tainted
ipv6 281069 12 - Live 0xbf02c000
spidev 5343 0 - Live 0xbf022000
joydev 9477 0 - Live 0xbf01c000
evdev 9486 0 - Live 0xbf015000
leds_gpio 2194 0 - Live 0xbf011000
led_class 3581 1 leds_gpio, Live 0xbf00d000
spi_bcm2708 4631 0 - Live 0xbf004000
i2c_bcm2708 3879 0 - Live 0xbf000000

Not working:
root@raspberrypi:~# lsmod
    Not tainted
ipv6 281069 12 - Live 0xbf00d000
joydev 9477 0 - Live 0xbf007000
evdev 9486 0 - Live 0xbf000000

At this point, if I then do the "ifdown"/"ifup" dance, my connection
works, but not more modules are loaded - does this help you guys?

--
Rich Bayliss

Re: [meta-raspberrypi] Network not working after first boot success

[Rich Bayliss]

On 18 July 2013 12:28, Rich Bayliss <richbayliss@gmail.com> wrote:
> On 18 July 2013 12:07, Paul Barker <paul@paulbarker.me.uk> wrote:
>> On 18 July 2013 11:55, Rich Bayliss <richbayliss@gmail.com> wrote:
>>>>>
>>>>> Is this a known issue? Does anyone have any ideas?
>>>>>
>>
>> Could you try running 'ip addr' and 'ip link' instead of 'ifconfig'?
>> Just wondering whether they query information differently and might
>> show something else. The other place to look is in the output of
>> 'dmesg'.
>>
>> Does un-plugging and re-plugging the network cable (without running
>> 'ifup' or 'ifdown') make any difference?
>>
>> --
>> Paul Barker
>>
>> Email: paul@paulbarker.me.uk
>> http://www.paulbarker.me.uk
>
> I have tried unplugging/replugging and it stays down.
>
> I have run the commands and there is a difference, however I also ran
> "lsmod" and there is a big difference.
>
> Working:
> root@raspberrypi:~# lsmod
>     Not tainted
> ipv6 281069 12 - Live 0xbf02c000
> spidev 5343 0 - Live 0xbf022000
> joydev 9477 0 - Live 0xbf01c000
> evdev 9486 0 - Live 0xbf015000
> leds_gpio 2194 0 - Live 0xbf011000
> led_class 3581 1 leds_gpio, Live 0xbf00d000
> spi_bcm2708 4631 0 - Live 0xbf004000
> i2c_bcm2708 3879 0 - Live 0xbf000000
>
> Not working:
> root@raspberrypi:~# lsmod
>     Not tainted
> ipv6 281069 12 - Live 0xbf00d000
> joydev 9477 0 - Live 0xbf007000
> evdev 9486 0 - Live 0xbf000000
>
> At this point, if I then do the "ifdown"/"ifup" dance, my connection
> works, but not more modules are loaded - does this help you guys?
>
> --
> Rich Bayliss

I have connected a serial terminal now, and I can provide boot logs.
It looks like maybe UDEV is causing/part of the issue. When it boots
successfully, then I dont get a "udevadm settle" message:

Starting udev
[    3.085091] smsc95xx v1.0.4
[    3.157196] smsc95xx 1-1.1:1.0: eth0: register 'smsc95xx' at
usb-bcm2708_usb-1.1, smsc95xx USB 2.0 Ethernet, b8:27:eb:e1:f6:e2
[    3.487588] udevd[74]: starting version 182
Starting Bootlog daemon: bootlogd.
[    6.162643] EXT4-fs (mmcblk0p2): re-mounted. Opts:
errors=remount-ro,data=ordered
Configuring network interfaces... udhcpc (v1.21.1) started
Sending discover...
[    9.138863] smsc95xx 1-1.1:1.0: eth0: link up, 100Mbps,
full-duplex, lpa 0xC1E1
Sending discover...
Sending select for 192.168.1.110...
Lease of 192.168.1.110 obtained, lease time 268435455
/etc/udhcpc.d/50default: Adding DNS 192.168.1.254
done.

However when the network is broken I see the following:

Starting udev
[    5.342406] udevd[74]: starting version 182
Starting Bootlog daemon: bootlogd.
[    7.938281] EXT4-fs (mmcblk0p2): re-mounted. Opts:
errors=remount-ro,data=ordered

udevadm settle - timeout of 3 seconds reached, the event queue contains:
  /sys/devices/platform/bcm2708_usb/usb1/1-1/1-1.1/1-1.1:1.0/net/eth0 (980)
  /sys/devices/platform/mmc_host/mmc0/mmc0:8fe4/block/mmcblk0/mmcblk0p1 (985)
  /sys/devices/platform/mmc_host/mmc0/mmc0:8fe4/block/mmcblk0/mmcblk0p2 (986)
Configuring network interfaces... ifup: interface lo already configured
ifup: interface eth0 already configured
done.

--
Rich Bayliss

Re: [meta-raspberrypi] Network not working after first boot success

[Rich Bayliss]

On 18 July 2013 14:50, Rich Bayliss <richbayliss@gmail.com> wrote:
> On 18 July 2013 12:28, Rich Bayliss <richbayliss@gmail.com> wrote:
>> On 18 July 2013 12:07, Paul Barker <paul@paulbarker.me.uk> wrote:
>>> On 18 July 2013 11:55, Rich Bayliss <richbayliss@gmail.com> wrote:
>>>>>>
>>>>>> Is this a known issue? Does anyone have any ideas?
>>>>>>
>>>
>>> Could you try running 'ip addr' and 'ip link' instead of 'ifconfig'?
>>> Just wondering whether they query information differently and might
>>> show something else. The other place to look is in the output of
>>> 'dmesg'.
>>>
>>> Does un-plugging and re-plugging the network cable (without running
>>> 'ifup' or 'ifdown') make any difference?
>>>
>>> --
>>> Paul Barker
>>>
>>> Email: paul@paulbarker.me.uk
>>> http://www.paulbarker.me.uk
>>
>> I have tried unplugging/replugging and it stays down.
>>
>> I have run the commands and there is a difference, however I also ran
>> "lsmod" and there is a big difference.
>>
>> Working:
>> root@raspberrypi:~# lsmod
>>     Not tainted
>> ipv6 281069 12 - Live 0xbf02c000
>> spidev 5343 0 - Live 0xbf022000
>> joydev 9477 0 - Live 0xbf01c000
>> evdev 9486 0 - Live 0xbf015000
>> leds_gpio 2194 0 - Live 0xbf011000
>> led_class 3581 1 leds_gpio, Live 0xbf00d000
>> spi_bcm2708 4631 0 - Live 0xbf004000
>> i2c_bcm2708 3879 0 - Live 0xbf000000
>>
>> Not working:
>> root@raspberrypi:~# lsmod
>>     Not tainted
>> ipv6 281069 12 - Live 0xbf00d000
>> joydev 9477 0 - Live 0xbf007000
>> evdev 9486 0 - Live 0xbf000000
>>
>> At this point, if I then do the "ifdown"/"ifup" dance, my connection
>> works, but not more modules are loaded - does this help you guys?
>>
>> --
>> Rich Bayliss
>
> I have connected a serial terminal now, and I can provide boot logs.
> It looks like maybe UDEV is causing/part of the issue. When it boots
> successfully, then I dont get a "udevadm settle" message:
>
> Starting udev
> [    3.085091] smsc95xx v1.0.4
> [    3.157196] smsc95xx 1-1.1:1.0: eth0: register 'smsc95xx' at
> usb-bcm2708_usb-1.1, smsc95xx USB 2.0 Ethernet, b8:27:eb:e1:f6:e2
> [    3.487588] udevd[74]: starting version 182
> Starting Bootlog daemon: bootlogd.
> [    6.162643] EXT4-fs (mmcblk0p2): re-mounted. Opts:
> errors=remount-ro,data=ordered
> Configuring network interfaces... udhcpc (v1.21.1) started
> Sending discover...
> [    9.138863] smsc95xx 1-1.1:1.0: eth0: link up, 100Mbps,
> full-duplex, lpa 0xC1E1
> Sending discover...
> Sending select for 192.168.1.110...
> Lease of 192.168.1.110 obtained, lease time 268435455
> /etc/udhcpc.d/50default: Adding DNS 192.168.1.254
> done.
>
> However when the network is broken I see the following:
>
> Starting udev
> [    5.342406] udevd[74]: starting version 182
> Starting Bootlog daemon: bootlogd.
> [    7.938281] EXT4-fs (mmcblk0p2): re-mounted. Opts:
> errors=remount-ro,data=ordered
>
> udevadm settle - timeout of 3 seconds reached, the event queue contains:
>   /sys/devices/platform/bcm2708_usb/usb1/1-1/1-1.1/1-1.1:1.0/net/eth0 (980)
>   /sys/devices/platform/mmc_host/mmc0/mmc0:8fe4/block/mmcblk0/mmcblk0p1 (985)
>   /sys/devices/platform/mmc_host/mmc0/mmc0:8fe4/block/mmcblk0/mmcblk0p2 (986)
> Configuring network interfaces... ifup: interface lo already configured
> ifup: interface eth0 already configured
> done.
>
> --
> Rich Bayliss

Could be a total wildcard, but I have managed to reliably reproduce
this behavior:

1) Boot from fresh SD image - network working.
2) run "dmesg | grep recovery" - no results
3) pull the power
4) Boots without network
5) run "dmesg | grep recovery" - 1 result "EXT4-fs (mmcblk0p2):
recovery complete"
6) run "reboot"
7) Boots with network
8) run "dmesg | grep recovery" - no results

I wonder if the dirty reboot by power pull as making the boot process
longer, and thus UDEV is failing to bring eth0 up in time? What do you
guys think?

--
Rich Bayliss

Re: [meta-raspberrypi] Network not working after first boot success

[Gary Thomas]

On 2013-07-18 08:07, Rich Bayliss wrote:
> On 18 July 2013 14:50, Rich Bayliss <richbayliss@gmail.com> wrote:
>> On 18 July 2013 12:28, Rich Bayliss <richbayliss@gmail.com> wrote:
>>> On 18 July 2013 12:07, Paul Barker <paul@paulbarker.me.uk> wrote:
>>>> On 18 July 2013 11:55, Rich Bayliss <richbayliss@gmail.com> wrote:
>>>>>>>
>>>>>>> Is this a known issue? Does anyone have any ideas?
>>>>>>>
>>>>
>>>> Could you try running 'ip addr' and 'ip link' instead of 'ifconfig'?
>>>> Just wondering whether they query information differently and might
>>>> show something else. The other place to look is in the output of
>>>> 'dmesg'.
>>>>
>>>> Does un-plugging and re-plugging the network cable (without running
>>>> 'ifup' or 'ifdown') make any difference?
>>>>
>>>> --
>>>> Paul Barker
>>>>
>>>> Email: paul@paulbarker.me.uk
>>>> http://www.paulbarker.me.uk
>>>
>>> I have tried unplugging/replugging and it stays down.
>>>
>>> I have run the commands and there is a difference, however I also ran
>>> "lsmod" and there is a big difference.
>>>
>>> Working:
>>> root@raspberrypi:~# lsmod
>>>      Not tainted
>>> ipv6 281069 12 - Live 0xbf02c000
>>> spidev 5343 0 - Live 0xbf022000
>>> joydev 9477 0 - Live 0xbf01c000
>>> evdev 9486 0 - Live 0xbf015000
>>> leds_gpio 2194 0 - Live 0xbf011000
>>> led_class 3581 1 leds_gpio, Live 0xbf00d000
>>> spi_bcm2708 4631 0 - Live 0xbf004000
>>> i2c_bcm2708 3879 0 - Live 0xbf000000
>>>
>>> Not working:
>>> root@raspberrypi:~# lsmod
>>>      Not tainted
>>> ipv6 281069 12 - Live 0xbf00d000
>>> joydev 9477 0 - Live 0xbf007000
>>> evdev 9486 0 - Live 0xbf000000
>>>
>>> At this point, if I then do the "ifdown"/"ifup" dance, my connection
>>> works, but not more modules are loaded - does this help you guys?
>>>
>>> --
>>> Rich Bayliss
>>
>> I have connected a serial terminal now, and I can provide boot logs.
>> It looks like maybe UDEV is causing/part of the issue. When it boots
>> successfully, then I dont get a "udevadm settle" message:
>>
>> Starting udev
>> [    3.085091] smsc95xx v1.0.4
>> [    3.157196] smsc95xx 1-1.1:1.0: eth0: register 'smsc95xx' at
>> usb-bcm2708_usb-1.1, smsc95xx USB 2.0 Ethernet, b8:27:eb:e1:f6:e2
>> [    3.487588] udevd[74]: starting version 182
>> Starting Bootlog daemon: bootlogd.
>> [    6.162643] EXT4-fs (mmcblk0p2): re-mounted. Opts:
>> errors=remount-ro,data=ordered
>> Configuring network interfaces... udhcpc (v1.21.1) started
>> Sending discover...
>> [    9.138863] smsc95xx 1-1.1:1.0: eth0: link up, 100Mbps,
>> full-duplex, lpa 0xC1E1
>> Sending discover...
>> Sending select for 192.168.1.110...
>> Lease of 192.168.1.110 obtained, lease time 268435455
>> /etc/udhcpc.d/50default: Adding DNS 192.168.1.254
>> done.
>>
>> However when the network is broken I see the following:
>>
>> Starting udev
>> [    5.342406] udevd[74]: starting version 182
>> Starting Bootlog daemon: bootlogd.
>> [    7.938281] EXT4-fs (mmcblk0p2): re-mounted. Opts:
>> errors=remount-ro,data=ordered
>>
>> udevadm settle - timeout of 3 seconds reached, the event queue contains:
>>    /sys/devices/platform/bcm2708_usb/usb1/1-1/1-1.1/1-1.1:1.0/net/eth0 (980)
>>    /sys/devices/platform/mmc_host/mmc0/mmc0:8fe4/block/mmcblk0/mmcblk0p1 (985)
>>    /sys/devices/platform/mmc_host/mmc0/mmc0:8fe4/block/mmcblk0/mmcblk0p2 (986)
>> Configuring network interfaces... ifup: interface lo already configured
>> ifup: interface eth0 already configured
>> done.
>>
>> --
>> Rich Bayliss
>
> Could be a total wildcard, but I have managed to reliably reproduce
> this behavior:
>
> 1) Boot from fresh SD image - network working.
> 2) run "dmesg | grep recovery" - no results
> 3) pull the power
> 4) Boots without network
> 5) run "dmesg | grep recovery" - 1 result "EXT4-fs (mmcblk0p2):
> recovery complete"
> 6) run "reboot"
> 7) Boots with network
> 8) run "dmesg | grep recovery" - no results
>
> I wonder if the dirty reboot by power pull as making the boot process
> longer, and thus UDEV is failing to bring eth0 up in time? What do you
> guys think?

I think the key difference is that when you type 'reboot' everything
is shut down sanely and the "saved state" indicates as such.  When you
simply yank the power, the state information is stale and incorrect,
causing your problems.

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: [meta-raspberrypi] Network not working after first boot success

[Rich Bayliss]

Indeed. However the usage requirement of the system rely on being
headless, and thus a power-pull is likely to happen.

I guess the only way to rule this in/out would be run with a RO rootfs
- and mount a sperate partition for user files. Do you know how I can
achieve this?

On 18 July 2013 15:13, Gary Thomas <gary@mlbassoc.com> wrote:
> On 2013-07-18 08:07, Rich Bayliss wrote:
>>
>> On 18 July 2013 14:50, Rich Bayliss <richbayliss@gmail.com> wrote:
>>>
>>> On 18 July 2013 12:28, Rich Bayliss <richbayliss@gmail.com> wrote:
>>>>
>>>> On 18 July 2013 12:07, Paul Barker <paul@paulbarker.me.uk> wrote:
>>>>>
>>>>> On 18 July 2013 11:55, Rich Bayliss <richbayliss@gmail.com> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> Is this a known issue? Does anyone have any ideas?
>>>>>>>>
>>>>>
>>>>> Could you try running 'ip addr' and 'ip link' instead of 'ifconfig'?
>>>>> Just wondering whether they query information differently and might
>>>>> show something else. The other place to look is in the output of
>>>>> 'dmesg'.
>>>>>
>>>>> Does un-plugging and re-plugging the network cable (without running
>>>>> 'ifup' or 'ifdown') make any difference?
>>>>>
>>>>> --
>>>>> Paul Barker
>>>>>
>>>>> Email: paul@paulbarker.me.uk
>>>>> http://www.paulbarker.me.uk
>>>>
>>>>
>>>> I have tried unplugging/replugging and it stays down.
>>>>
>>>> I have run the commands and there is a difference, however I also ran
>>>> "lsmod" and there is a big difference.
>>>>
>>>> Working:
>>>> root@raspberrypi:~# lsmod
>>>>      Not tainted
>>>> ipv6 281069 12 - Live 0xbf02c000
>>>> spidev 5343 0 - Live 0xbf022000
>>>> joydev 9477 0 - Live 0xbf01c000
>>>> evdev 9486 0 - Live 0xbf015000
>>>> leds_gpio 2194 0 - Live 0xbf011000
>>>> led_class 3581 1 leds_gpio, Live 0xbf00d000
>>>> spi_bcm2708 4631 0 - Live 0xbf004000
>>>> i2c_bcm2708 3879 0 - Live 0xbf000000
>>>>
>>>> Not working:
>>>> root@raspberrypi:~# lsmod
>>>>      Not tainted
>>>> ipv6 281069 12 - Live 0xbf00d000
>>>> joydev 9477 0 - Live 0xbf007000
>>>> evdev 9486 0 - Live 0xbf000000
>>>>
>>>> At this point, if I then do the "ifdown"/"ifup" dance, my connection
>>>> works, but not more modules are loaded - does this help you guys?
>>>>
>>>> --
>>>> Rich Bayliss
>>>
>>>
>>> I have connected a serial terminal now, and I can provide boot logs.
>>> It looks like maybe UDEV is causing/part of the issue. When it boots
>>> successfully, then I dont get a "udevadm settle" message:
>>>
>>> Starting udev
>>> [    3.085091] smsc95xx v1.0.4
>>> [    3.157196] smsc95xx 1-1.1:1.0: eth0: register 'smsc95xx' at
>>> usb-bcm2708_usb-1.1, smsc95xx USB 2.0 Ethernet, b8:27:eb:e1:f6:e2
>>> [    3.487588] udevd[74]: starting version 182
>>> Starting Bootlog daemon: bootlogd.
>>> [    6.162643] EXT4-fs (mmcblk0p2): re-mounted. Opts:
>>> errors=remount-ro,data=ordered
>>> Configuring network interfaces... udhcpc (v1.21.1) started
>>> Sending discover...
>>> [    9.138863] smsc95xx 1-1.1:1.0: eth0: link up, 100Mbps,
>>> full-duplex, lpa 0xC1E1
>>> Sending discover...
>>> Sending select for 192.168.1.110...
>>> Lease of 192.168.1.110 obtained, lease time 268435455
>>> /etc/udhcpc.d/50default: Adding DNS 192.168.1.254
>>> done.
>>>
>>> However when the network is broken I see the following:
>>>
>>> Starting udev
>>> [    5.342406] udevd[74]: starting version 182
>>> Starting Bootlog daemon: bootlogd.
>>> [    7.938281] EXT4-fs (mmcblk0p2): re-mounted. Opts:
>>> errors=remount-ro,data=ordered
>>>
>>> udevadm settle - timeout of 3 seconds reached, the event queue contains:
>>>    /sys/devices/platform/bcm2708_usb/usb1/1-1/1-1.1/1-1.1:1.0/net/eth0
>>> (980)
>>>    /sys/devices/platform/mmc_host/mmc0/mmc0:8fe4/block/mmcblk0/mmcblk0p1
>>> (985)
>>>    /sys/devices/platform/mmc_host/mmc0/mmc0:8fe4/block/mmcblk0/mmcblk0p2
>>> (986)
>>> Configuring network interfaces... ifup: interface lo already configured
>>> ifup: interface eth0 already configured
>>> done.
>>>
>>> --
>>> Rich Bayliss
>>
>>
>> Could be a total wildcard, but I have managed to reliably reproduce
>> this behavior:
>>
>> 1) Boot from fresh SD image - network working.
>> 2) run "dmesg | grep recovery" - no results
>> 3) pull the power
>> 4) Boots without network
>> 5) run "dmesg | grep recovery" - 1 result "EXT4-fs (mmcblk0p2):
>> recovery complete"
>> 6) run "reboot"
>> 7) Boots with network
>> 8) run "dmesg | grep recovery" - no results
>>
>> I wonder if the dirty reboot by power pull as making the boot process
>> longer, and thus UDEV is failing to bring eth0 up in time? What do you
>> guys think?
>
>
> I think the key difference is that when you type 'reboot' everything
> is shut down sanely and the "saved state" indicates as such.  When you
> simply yank the power, the state information is stale and incorrect,
> causing your problems.
>
>
> --
> ------------------------------------------------------------
> Gary Thomas                 |  Consulting for the
> MLB Associates              |    Embedded world
> ------------------------------------------------------------
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto



-- 
Rich Bayliss

Re: [meta-raspberrypi] Network not working after first boot success

[Gary Thomas]

On 2013-07-18 08:20, Rich Bayliss wrote:
> Indeed. However the usage requirement of the system rely on being
> headless, and thus a power-pull is likely to happen.
>
> I guess the only way to rule this in/out would be run with a RO rootfs
> - and mount a sperate partition for user files. Do you know how I can
> achieve this?

There is some support in recent OE-core for read-only rootfs.  Search
the Yocto project archives for more information (I've not tried it yet).

The more interesting thing is that I have systems other than the RPI
that are built using Poky/Yocto where the power is often the cause
of a reboot and I've never seen this error/situation occur!  Maybe
I can experiment a bit and compare the RPI build with my other systems
(I have both) and see why.  Not sure when I can get to it though...

>
> On 18 July 2013 15:13, Gary Thomas <gary@mlbassoc.com> wrote:
>> On 2013-07-18 08:07, Rich Bayliss wrote:
>>>
>>> On 18 July 2013 14:50, Rich Bayliss <richbayliss@gmail.com> wrote:
>>>>
>>>> On 18 July 2013 12:28, Rich Bayliss <richbayliss@gmail.com> wrote:
>>>>>
>>>>> On 18 July 2013 12:07, Paul Barker <paul@paulbarker.me.uk> wrote:
>>>>>>
>>>>>> On 18 July 2013 11:55, Rich Bayliss <richbayliss@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Is this a known issue? Does anyone have any ideas?
>>>>>>>>>
>>>>>>
>>>>>> Could you try running 'ip addr' and 'ip link' instead of 'ifconfig'?
>>>>>> Just wondering whether they query information differently and might
>>>>>> show something else. The other place to look is in the output of
>>>>>> 'dmesg'.
>>>>>>
>>>>>> Does un-plugging and re-plugging the network cable (without running
>>>>>> 'ifup' or 'ifdown') make any difference?
>>>>>>
>>>>>> --
>>>>>> Paul Barker
>>>>>>
>>>>>> Email: paul@paulbarker.me.uk
>>>>>> http://www.paulbarker.me.uk
>>>>>
>>>>>
>>>>> I have tried unplugging/replugging and it stays down.
>>>>>
>>>>> I have run the commands and there is a difference, however I also ran
>>>>> "lsmod" and there is a big difference.
>>>>>
>>>>> Working:
>>>>> root@raspberrypi:~# lsmod
>>>>>       Not tainted
>>>>> ipv6 281069 12 - Live 0xbf02c000
>>>>> spidev 5343 0 - Live 0xbf022000
>>>>> joydev 9477 0 - Live 0xbf01c000
>>>>> evdev 9486 0 - Live 0xbf015000
>>>>> leds_gpio 2194 0 - Live 0xbf011000
>>>>> led_class 3581 1 leds_gpio, Live 0xbf00d000
>>>>> spi_bcm2708 4631 0 - Live 0xbf004000
>>>>> i2c_bcm2708 3879 0 - Live 0xbf000000
>>>>>
>>>>> Not working:
>>>>> root@raspberrypi:~# lsmod
>>>>>       Not tainted
>>>>> ipv6 281069 12 - Live 0xbf00d000
>>>>> joydev 9477 0 - Live 0xbf007000
>>>>> evdev 9486 0 - Live 0xbf000000
>>>>>
>>>>> At this point, if I then do the "ifdown"/"ifup" dance, my connection
>>>>> works, but not more modules are loaded - does this help you guys?
>>>>>
>>>>> --
>>>>> Rich Bayliss
>>>>
>>>>
>>>> I have connected a serial terminal now, and I can provide boot logs.
>>>> It looks like maybe UDEV is causing/part of the issue. When it boots
>>>> successfully, then I dont get a "udevadm settle" message:
>>>>
>>>> Starting udev
>>>> [    3.085091] smsc95xx v1.0.4
>>>> [    3.157196] smsc95xx 1-1.1:1.0: eth0: register 'smsc95xx' at
>>>> usb-bcm2708_usb-1.1, smsc95xx USB 2.0 Ethernet, b8:27:eb:e1:f6:e2
>>>> [    3.487588] udevd[74]: starting version 182
>>>> Starting Bootlog daemon: bootlogd.
>>>> [    6.162643] EXT4-fs (mmcblk0p2): re-mounted. Opts:
>>>> errors=remount-ro,data=ordered
>>>> Configuring network interfaces... udhcpc (v1.21.1) started
>>>> Sending discover...
>>>> [    9.138863] smsc95xx 1-1.1:1.0: eth0: link up, 100Mbps,
>>>> full-duplex, lpa 0xC1E1
>>>> Sending discover...
>>>> Sending select for 192.168.1.110...
>>>> Lease of 192.168.1.110 obtained, lease time 268435455
>>>> /etc/udhcpc.d/50default: Adding DNS 192.168.1.254
>>>> done.
>>>>
>>>> However when the network is broken I see the following:
>>>>
>>>> Starting udev
>>>> [    5.342406] udevd[74]: starting version 182
>>>> Starting Bootlog daemon: bootlogd.
>>>> [    7.938281] EXT4-fs (mmcblk0p2): re-mounted. Opts:
>>>> errors=remount-ro,data=ordered
>>>>
>>>> udevadm settle - timeout of 3 seconds reached, the event queue contains:
>>>>     /sys/devices/platform/bcm2708_usb/usb1/1-1/1-1.1/1-1.1:1.0/net/eth0
>>>> (980)
>>>>     /sys/devices/platform/mmc_host/mmc0/mmc0:8fe4/block/mmcblk0/mmcblk0p1
>>>> (985)
>>>>     /sys/devices/platform/mmc_host/mmc0/mmc0:8fe4/block/mmcblk0/mmcblk0p2
>>>> (986)
>>>> Configuring network interfaces... ifup: interface lo already configured
>>>> ifup: interface eth0 already configured
>>>> done.
>>>>
>>>> --
>>>> Rich Bayliss
>>>
>>>
>>> Could be a total wildcard, but I have managed to reliably reproduce
>>> this behavior:
>>>
>>> 1) Boot from fresh SD image - network working.
>>> 2) run "dmesg | grep recovery" - no results
>>> 3) pull the power
>>> 4) Boots without network
>>> 5) run "dmesg | grep recovery" - 1 result "EXT4-fs (mmcblk0p2):
>>> recovery complete"
>>> 6) run "reboot"
>>> 7) Boots with network
>>> 8) run "dmesg | grep recovery" - no results
>>>
>>> I wonder if the dirty reboot by power pull as making the boot process
>>> longer, and thus UDEV is failing to bring eth0 up in time? What do you
>>> guys think?
>>
>>
>> I think the key difference is that when you type 'reboot' everything
>> is shut down sanely and the "saved state" indicates as such.  When you
>> simply yank the power, the state information is stale and incorrect,
>> causing your problems.
>>
>>
>> --
>> ------------------------------------------------------------
>> Gary Thomas                 |  Consulting for the
>> MLB Associates              |    Embedded world
>> ------------------------------------------------------------
>> _______________________________________________
>> yocto mailing list
>> yocto@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/yocto
>
>
>

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: [meta-raspberrypi] Network not working after first boot success

[Rich Bayliss]

On 18 July 2013 15:38, Gary Thomas <gary@mlbassoc.com> wrote:
> On 2013-07-18 08:20, Rich Bayliss wrote:
>>
>> Indeed. However the usage requirement of the system rely on being
>> headless, and thus a power-pull is likely to happen.
>>
>> I guess the only way to rule this in/out would be run with a RO rootfs
>> - and mount a sperate partition for user files. Do you know how I can
>> achieve this?
>
>
> There is some support in recent OE-core for read-only rootfs.  Search
> the Yocto project archives for more information (I've not tried it yet).
>
> The more interesting thing is that I have systems other than the RPI
> that are built using Poky/Yocto where the power is often the cause
> of a reboot and I've never seen this error/situation occur!  Maybe
> I can experiment a bit and compare the RPI build with my other systems
> (I have both) and see why.  Not sure when I can get to it though...
> ------------------------------------------------------------
> Gary Thomas                 |  Consulting for the
> MLB Associates              |    Embedded world
> ------------------------------------------------------------

OK Gary, thanks. I have found an article which states including
"read-only-rootfs" in the IMAGE_FEATURES will provide me with just
that. I am building now, so let's see what happens.

In my opinion, although a power-failure reboot is undesirable from the
OS perspective, I would expect it to come back with networking up - or
at least auto-magically reboot if they aren't for some reason.

--
Rich Bayliss

Re: [meta-raspberrypi] Network not working after first boot success

[Andrei Gherzan]

[-- Attachment #1: Type: text/plain, Size: 1982 bytes --]

On Jul 18, 2013 5:42 PM, "Rich Bayliss" <richbayliss@gmail.com> wrote:
>
> On 18 July 2013 15:38, Gary Thomas <gary@mlbassoc.com> wrote:
> > On 2013-07-18 08:20, Rich Bayliss wrote:
> >>
> >> Indeed. However the usage requirement of the system rely on being
> >> headless, and thus a power-pull is likely to happen.
> >>
> >> I guess the only way to rule this in/out would be run with a RO rootfs
> >> - and mount a sperate partition for user files. Do you know how I can
> >> achieve this?
> >
> >
> > There is some support in recent OE-core for read-only rootfs.  Search
> > the Yocto project archives for more information (I've not tried it yet).
> >
> > The more interesting thing is that I have systems other than the RPI
> > that are built using Poky/Yocto where the power is often the cause
> > of a reboot and I've never seen this error/situation occur!  Maybe
> > I can experiment a bit and compare the RPI build with my other systems
> > (I have both) and see why.  Not sure when I can get to it though...
> > ------------------------------------------------------------
> > Gary Thomas                 |  Consulting for the
> > MLB Associates              |    Embedded world
> > ------------------------------------------------------------
>
> OK Gary, thanks. I have found an article which states including
> "read-only-rootfs" in the IMAGE_FEATURES will provide me with just
> that. I am building now, so let's see what happens.
>
> In my opinion, although a power-failure reboot is undesirable from the
> OS perspective, I would expect it to come back with networking up - or
> at least auto-magically reboot if they aren't for some reason.

I agree. It should be running correctly. And if it doesn't we need to fix
this. Power fails happen. So this is no excuse.

Well, it seems that I can't replicate this. Have network all the time. 4
reboots. Will retry tomorrow. Don't know what build I have. Will recompile
master.

ag

[-- Attachment #2: Type: text/html, Size: 2508 bytes --]

Re: [meta-raspberrypi] Network not working after first boot success

[Eric Bénard]

Hi Rich,

Le Thu, 18 Jul 2013 15:07:01 +0100,
Rich Bayliss <richbayliss@gmail.com> a écrit :

> On 18 July 2013 14:50, Rich Bayliss <richbayliss@gmail.com> wrote:
> > Configuring network interfaces... ifup: interface lo already configured
> > ifup: interface eth0 already configured
> > done.
> >
> I wonder if the dirty reboot by power pull as making the boot process
> longer, and thus UDEV is failing to bring eth0 up in time? What do you
> guys think?
> 
this problem occured here when /var/run/network/ifstate is on a RW
persistent storage and not in a tmpfs : the interface state can exist
when booting after a dirty reboot.

A workaround is to hack
meta/recipes-core/init-ifupdown/init-ifupdown-1.0/init and add
ifdown -a
just before ifup -a in the start) section

Eric

Re: [meta-raspberrypi] Network not working after first boot success

[Rich Bayliss]

On 18 July 2013 23:27, Eric Bénard <eric@eukrea.com> wrote:

> this problem occured here when /var/run/network/ifstate is on a RW
> persistent storage and not in a tmpfs : the interface state can exist
> when booting after a dirty reboot.
>
> A workaround is to hack
> meta/recipes-core/init-ifupdown/init-ifupdown-1.0/init and add
> ifdown -a
> just before ifup -a in the start) section
>
> Eric

I will try this hack Eric, but I suppose this opens another question -
why /var/run is held on a non-TMPFS? I would think (and this is not
based on any authority) that since /var/run is mainly for tracking
state, that after a reboot of any kind it should remain blank.

I have tested deleting /var/run/ifstate and pulling the plug, and it
returns on next reboot. I think this might be the EXT4 recovery
kicking in and putting the file back?

--
Rich Bayliss

Re: [meta-raspberrypi] Network not working after first boot success

[Gary Thomas]

On 2013-07-18 07:50, Rich Bayliss wrote:
> On 18 July 2013 12:28, Rich Bayliss <richbayliss@gmail.com> wrote:
>> On 18 July 2013 12:07, Paul Barker <paul@paulbarker.me.uk> wrote:
>>> On 18 July 2013 11:55, Rich Bayliss <richbayliss@gmail.com> wrote:
>>>>>>
>>>>>> Is this a known issue? Does anyone have any ideas?
>>>>>>
>>>
>>> Could you try running 'ip addr' and 'ip link' instead of 'ifconfig'?
>>> Just wondering whether they query information differently and might
>>> show something else. The other place to look is in the output of
>>> 'dmesg'.
>>>
>>> Does un-plugging and re-plugging the network cable (without running
>>> 'ifup' or 'ifdown') make any difference?
>>>
>>> --
>>> Paul Barker
>>>
>>> Email: paul@paulbarker.me.uk
>>> http://www.paulbarker.me.uk
>>
>> I have tried unplugging/replugging and it stays down.
>>
>> I have run the commands and there is a difference, however I also ran
>> "lsmod" and there is a big difference.
>>
>> Working:
>> root@raspberrypi:~# lsmod
>>      Not tainted
>> ipv6 281069 12 - Live 0xbf02c000
>> spidev 5343 0 - Live 0xbf022000
>> joydev 9477 0 - Live 0xbf01c000
>> evdev 9486 0 - Live 0xbf015000
>> leds_gpio 2194 0 - Live 0xbf011000
>> led_class 3581 1 leds_gpio, Live 0xbf00d000
>> spi_bcm2708 4631 0 - Live 0xbf004000
>> i2c_bcm2708 3879 0 - Live 0xbf000000
>>
>> Not working:
>> root@raspberrypi:~# lsmod
>>      Not tainted
>> ipv6 281069 12 - Live 0xbf00d000
>> joydev 9477 0 - Live 0xbf007000
>> evdev 9486 0 - Live 0xbf000000
>>
>> At this point, if I then do the "ifdown"/"ifup" dance, my connection
>> works, but not more modules are loaded - does this help you guys?
>>
>> --
>> Rich Bayliss
>
> I have connected a serial terminal now, and I can provide boot logs.
> It looks like maybe UDEV is causing/part of the issue. When it boots
> successfully, then I dont get a "udevadm settle" message:
>
> Starting udev
> [    3.085091] smsc95xx v1.0.4
> [    3.157196] smsc95xx 1-1.1:1.0: eth0: register 'smsc95xx' at
> usb-bcm2708_usb-1.1, smsc95xx USB 2.0 Ethernet, b8:27:eb:e1:f6:e2
> [    3.487588] udevd[74]: starting version 182
> Starting Bootlog daemon: bootlogd.
> [    6.162643] EXT4-fs (mmcblk0p2): re-mounted. Opts:
> errors=remount-ro,data=ordered
> Configuring network interfaces... udhcpc (v1.21.1) started
> Sending discover...
> [    9.138863] smsc95xx 1-1.1:1.0: eth0: link up, 100Mbps,
> full-duplex, lpa 0xC1E1
> Sending discover...
> Sending select for 192.168.1.110...
> Lease of 192.168.1.110 obtained, lease time 268435455
> /etc/udhcpc.d/50default: Adding DNS 192.168.1.254
> done.
>
> However when the network is broken I see the following:
>
> Starting udev
> [    5.342406] udevd[74]: starting version 182
> Starting Bootlog daemon: bootlogd.
> [    7.938281] EXT4-fs (mmcblk0p2): re-mounted. Opts:
> errors=remount-ro,data=ordered
>
> udevadm settle - timeout of 3 seconds reached, the event queue contains:
>    /sys/devices/platform/bcm2708_usb/usb1/1-1/1-1.1/1-1.1:1.0/net/eth0 (980)
>    /sys/devices/platform/mmc_host/mmc0/mmc0:8fe4/block/mmcblk0/mmcblk0p1 (985)
>    /sys/devices/platform/mmc_host/mmc0/mmc0:8fe4/block/mmcblk0/mmcblk0p2 (986)
> Configuring network interfaces... ifup: interface lo already configured
> ifup: interface eth0 already configured
> done.
>

This looks more like there is some state information left around
that is keeping the network from coming up - it thinks that it
already is up.

In your original post, you mentioned 'I pull the power/reboot'.  Is that
how you are rebooting the board, by just yanking the power?  If so, that
could definitely explain such a case where state information says one
thing and real life doesn't match.

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------