All of lore.kernel.org
 help / color / mirror / Atom feed
* Sparse 0.5.1 RC5 released.
@ 2017-08-12 14:15 Christopher Li
  2017-08-12 20:26 ` Michael Stefaniuc
                   ` (2 more replies)
  0 siblings, 3 replies; 63+ messages in thread
From: Christopher Li @ 2017-08-12 14:15 UTC (permalink / raw)
  To: Linux-Sparse
  Cc: Luc Van Oostenryck, Ramsay Jones, Lance Richardson, Michael Stefaniuc

It took longer than I thought. But we are finally there.

I am very happy that the wine dead loop bug is nailed
down and the looping while deleting bugs are stopped.

Thanks Luc for a lot of hard work there.

I hope this is the last RC for this release.

Let's ship this release out. The work for the next release
is already start to piling up.

Thanks every one that contribute to this RC5.

Again, please give it a good test.  Report back
any issue you might have.


Chris




Aaro Koskinen (1):
      build: allow use of PKG_CONFIG to override pkg-config

Andy Shevchenko (1):
      lib.c: skip --param parameters

Ard Biesheuvel (2):
      sparse: treat function pointers as pointers to const data
      Ignore pure attribute in assignement

Azat Khuzhin (2):
      sparse, llvm: compile: skip function prototypes to avoid SIGSEGV
      validation/prototype: regression for skipping prototypes

Christian Borntraeger (1):
      s390x: add the proper defines for data types

Christopher Li (23):
      Minor clean up for option handling
      round up the array element size to byte align
      Make same_symbol list share the same scope
      rename -Werror to -Wsparse-error
      teach next_designators() use array_element_offset()
      Ptr list sorting should use memmove instead of memcpy
      Make macro expanded string immutable
      Fix warning compiling sparse-llvm
      Adding ignored attribute optimize
      Let create_symbol check for previous same symbol
      Add full list of gcc attribute
      bump sparse's version to 0.5.1-rc4
      Adding gcc attribute no_gccisr
      Add test case for the wine dead loop bug
      Makefile: clean up and simplify
      Makefile: add selfcheck target
      Adding _Pragma()
      fix warnings report by selfcheck
      Adding gcc attribute noipa etc
      Adding document for sparse patch submit process
      Documents: project ideas
      test-inspect: handle special case iter==NULL
      test-inspect: Detect gtk3 then gtk2 package

Cody P Schafer (3):
      build: allow use of LLVM_CONFIG to override llvm-config config script
      sparse{i,c}: use LLVM_CONFIG to find llc and lli
      parse: support c99 [static ...] in abstract array declarators

Dan Carpenter (1):
      ptrlist: reading deleted items in NEXT_PTR_LIST()

Daniel Wagner (1):
      parse: Add comment to struct statement

Edward Cree (1):
      Allow casting to a restricted type if !restricted_value

Emilio G. Cota (1):
      Define __CHAR_BIT__

Emily Maier (2):
      linearize: Emit C99 declarations correctly
      validation: Check C99 for loop variables

Hans Verkuil (3):
      Add test case for extern array
      Add test case for anonymous union initializer
      Add test case for the ioc type check

Heiko Carstens (1):
      sparse/parse.c: ignore hotpatch attribute

Jeff Layton (2):
      sparse: make bits_to_bytes round up instead of down
      Handle SForced in storage_modifiers

Joe Perches (1):
      sparse: Allow override of sizeof(bool) warning

Johannes Berg (1):
      implement constant-folding in __builtin_bswap*()

John Keeping (3):
      validation/sizeof-bool: fix broken test case
      evaluate: split out implementation of compatible_assignment_types
      Support GCC's transparent unions

Lance Richardson (3):
      sparse: ignore __assume_aligned__ attribute
      sparse: update __builtin_object_size() prototype
      sparse: add support for _Static_assert

Linus Torvalds (5):
      Add warning about duplicate initializers
      Use any previous initializer to size a symbol
      Fix error at anoymous unions
      Fix scoping of extern symbols in block scope
      Fix initializers in anonymous structs and unions

Luc Van Oostenryck (241):
      Teach sparse about the __COUNTER__ predefined macro
      Fix size calculation of unsized bool array
      Do not drop 'nocast' modifier when taking the address.
      fix mixup in "Handle SForced in storage_modifiers"
      Fix type checking of variadic functions
      add missing #include "char.h" to char.c
      make 'ignored_attributes[]' static
      cleanup: remove evaluate_arguments()'s unused argument
      Warn on unknown attributes instead of throwing errors
      Remove unneeded variable in integer_promotion()
      fix discarded label statement
      add test case for builtin bswap with constant args
      make ptrlist walking against robust against empty blocks
      let "compile" not crash on bools
      give comparable label's names to basic blocks
      OP_SWITCH should use 'insn->cond' instead of 'insn->target'
      remove unused field 'multijump' in struct instruction
      storage should not be inherited by pointers
      testsuite: simplify test function-pointer-inheritance
      use a shorter name for function-pointer-modifier-inheritance.c
      testsuite: test modifiers preserved by '&' operator
      testsuite: test modifiers preserved by 'typeof()'
      some modifiers need to be preserved by 'typeof()'
      Update maintainers in the manpage
      cgcc should not define non-reserved identifiers
      recursive phi_defines cannot happen
      fix missing element in types declaration
      add support for __int128
      fix typing error in compound assignment
      llvm: fix typing when comparing to a constant
      llvm: remove unneeded OP_COPY support
      fix cast to bool
      unssa: do not try to update liveness
      unssa: simplify rewrite of OP_PHISOURCE
      unssa: try to avoid some OP_PHI copies
      unssa: eliminate trivial phisrc copies
      unssa: update comment about the unneeded copies
      volatile loads must not be simplified
      fix superfluous phisrc
      fix phisrc mixup
      missing load simplification
      fix value of label statement
      C11: teach sparse about '_Thread_local'
      C11: teach sparse about '_Noreturn'
      C11: teach sparse about '_Alignof()'
      C11: teach sparse about '_Alignas()'
      C11: teach sparse about '--std={c11,gnu11}'
      fix cast's target type info
      fix crash while testing between conditional & unconditional OP_BR
      kill uses of replaced instructions
      fix killing OP_PHI instructions
      fix killing OP_CAST & friends
      fix killing OP_SELECT
      fix killing OP_COMPUTEDGOTO
      explicitely ignore killing OP_ENTRY
      cleanup kill_instruction()
      fix conditional context test case with void
      add helper: is_scalar_type()
      validate expression's type in conditionals
      remove unused arg in uses/defs functions
      add testcase for wrong early escape conversion
      warn on unknown escapes after preprocessing
      remove 'Escape' from token character class
      fix killing OP_SETVAL instructions
      define __LP64__ & _LP64 if arch_m64 is enabled
      add an helper for common predefined macros
      define __LONG_MAX__ & __SIZEOF_POINTER__
      move OP_MUL simplification in a separate function
      simplify '(x / 1)' to 'x'
      simplify '(x * -1)' to '-x'
      simplify '(x / -1)' to '-x' (but only for signed division)
      simplify '(x % 1)' into '0'
      simplify '~(~x)' and '-(-x)' to 'x'
      simplify '(x || 1)' to '1'
      simplify '(x op x)' to '0', '1' or 'x'
      add warning option '-Wtautological-compare'
      simplify comparisons followed by an equality test against 0 or 1
      simplify '(x || x)' and '(x && x)'
      add support for LLP64 arch
      move evaluation & expansion of builtins in a separate file
      let identical symbols share their evaluate/expand methods
      expand __builtin_bswap*() with constant args
      testsuite: give a proper name to the 'binary-constant' test
      testsuite: make tests known to fail effectively fail
      testsuite: simplify the ioc-typecheck case
      testsuite: add a simple test for -Wenum-mismatch
      testsuite: add tag to ignore the output/error
      testsuite: report as error tests known to fail but which succeed
      allow to launch the test suite from the project root dir
      testsuite: check patterns presence or absence in output
      testsuite: add some selfchecking
      testsuite: check the nbr of times a pattern should be present
      testsuite: use 'error' instead of 'info' for successful tests
known to fail
      testsuite: get 'check-known-to-fail' earlier
      testsuite: allow quieter error reporting
      testsuite: quieter error reporting for 'known-to-fail'
      cleanup: there is no 'struct phi' to allocate
      remove unused field 'multijmp' in struct statement
      remove unused field 'goto_bb' in struct statement
      fix show-parse()'s labels
      add killing of OP_SLICEs
      add killing of OP_PHISOURCEs
      add helper kill_use_list()
      fix killing of OP_PHIs
      fix clear_phi(), replace it by kill_instruction()
      remove unused clear_phi()
      fix killing of otherwise not-handled instructions
      kill_instruction() may need to be forced or not
      add killing of pure calls
      fix killing OP_CALL via pointers
      add killing of non-volatile loads
      add killing of stores
      fix killing of rewritten loads
      use kill_instruction() when killing an OP_PHI during CSE
      use kill_instruction() when killing any instructions during CSE
      fix OP_PHI usage in try_to_simplify_bb()
      simplify float-to-float casts that doesn't change size
      CSE: add test cases for comparisons duality
      CSE: use commutativity to identify equivalent instructions
      CSE: avoid hashing removed instructions
      fix expansion cost of pure functions
      add missing braces around FOR_EACH_PTR loop
      make -Wbitwise operational again
      use option: '-Woverride-init'
      add test case for warnings about overlapping initializers
      allow to warn on all overlapping initializers
      fix checking of overlapping initializer
      ignore whole-range overlapping initializer
      fix usage in simplify_seteq_setne()
      fix size of loaded bitfields
      split OP_BR between unconditional & conditional: OP_CBR
      remove unused helper is_branch_goto()
      replace test for c99 for-loop initializers
      add test case for scope of C99 for-loop declarations
      add test cases for storage of c99 for-loop declarations
      add an optional validation method to external_declaration()
      check the storage of C99 for-loop initializers
      move 'extern with initializer' validation after the validate method
      use VOID instead of directly using &void_pseudo
      teach sparse about -Waddress
      add is_func_type()
      warn if testing the address of a function
      add is_array_type()
      warn if testing the address of an array
      fix evaluation of a function or array symbol in conditionals
      fix is_scalar_type()
      fix test for cast to bool on 32bit machines
      predefine __INT_MAX__ and friends
      predefine __SIZEOF_INT__ & friends
      fix test validation/div.c
      fix cast to pointer to floating-point
      do not depends on limits.h to test __CHAR_BIT__
      fix expansion of integers to floats
      avoid crash with test-linearize -vv
      fix OP_PHI usage in try_to_simplify_bb(), correctly
      be more careful with concat_user_list()
      avoid useless warning for 'bool <- restricted type' conversion
      introduce REPEAT_CFG_CLEANUP
      let kill_unreachable_bbs() clear REPEAT_CFG_CLEANUP
      fix: kill unreachable BBs after killing a child
      ignore VOID when trying to if-convert phi-nodes
      fix boolean context for OP_AND_BOOL & OP_OR_BOOL
      fix missing reload
      keyword: add test case for reserved '_Static_assert'
      keyword: regroup the [reserved] keywords
      keyword: explicitly add C99 & C11 keywords
      keyword: add more reserved keywords to the test case
      keyword: add a comment about NS_TYPEDEF & reserved keywords
      keyword: no pre-declaration needed for attribute names
      add get_<allocator>_stats()
      add show_allocation_stats()
      add helper handle_simple_switch()
      teach sparse how to handle '-fmem-report'
      use -fmem-report to report allocation stats
      testsuite: cleanup result files
      fix: kill old branch in insert_branch()
      returns the correct type when evaluating NULL
      remove bit_size & bit_offset from struct access_data
      add test case for linearize_initializer() of bitfields
      fix implicit zero initializer.
      remove alignment from struct access_data
      remove origval from struct access_data
      add support for a new flag: -fdump-linearize[=only]
      more tests for implicit 'bool <- restricted' casts
      avoid warning on explicit 'bool <- restricted' casts
      define ident_list
      teach sparse how to dump macro definitions
      fix hardcoded size of wide chars
      avoid to redefine __INT_MAX__ and friends
      fix definition of __SCHAR_MAX__ & friends
      teach sparse how to handle -dD flag
      let -dD report macro definitions
      testsuite: get all tags in once
      testsuite: grep the expected output only when needed
      testsuite: grep the output patterns only when needed
      testsuite: use shell arithmetic instead of fork-execing expr
      testsuite: remove unneeded './' before commands
      testsuite: avoid fork+execing basename
      teach cgcc about OSX aka darwin
      ret-void: add test case for toplevel asm
      ret-void: warn for implicit type
      use NULL instead of 0 in testcases.
      finer control over error vs. warnings
      Add more declarations for more builtin functions
      keep the warnings table alphabetically sorted
      cgcc: alphasort warning names in check_only_option()
      cgcc: add missing warning names to check_only_option()
      cgcc: filter-out '-fdump-linearize[=...]'
      memcpy()'s byte count is unsigned
      add support for -Wmemcpy-max-count
      add support for -fmemcpy-max-count
      fix: add missing examine in evaluate_dereference()
      fix OP_PHI usage in try_to_simplify_bb() only when non-bogus
      fix: try_to_simplify_bb eargerness
      add fallback for missing __builtin_bswapXX()
      fix: __builtin_bswap{16,32,64}() constantness
      dissect: use built_in_ident() instead of MK_IDENT()
      teach sparse about -m{big,little}-endian
      teach sparse about __{BIG,LITTLE}_ENDIAN__
      teach sparse about __BYTE_ORDER__ & __ORDER_{BIG,LITTLE}_ENDIAN__
      cgcc: teach cgcc about arm64
      cgcc: teach cgcc about ppc64[le]
      cgcc: teach cgcc about arm
      bump sparse's version to -rc3
      fix ptrlist corruption while killing unreachable BBs
      fix infinite simplification loops
      fix BB dependencies on phi-nodes
      fix crash when ep->active is NULL
      fix crash in rewrite_branch()
      fix some crashes in add_dominators()
      fix crash with sym->bb_target == NULL
      take comma expr in account for constant value
      fix: give a type to bad cond expr with known condition
      ptrlist: add a counter for the number of removed elemnets
      ptrlist: adjust ptr_list_size for the new ->rm field
      ptrlist: add MARK_CURRENT_DELETED
      ptrlist: avoid iteration on NULL entries
      mark pseudo users as deleted instead of removing them
      testsuite: add support for commands with timeout
      Remove single-store shortcut
      Bump sparse's version to -rc5

Michael Stefaniuc (3):
      Add the __builtin functions needed for INFINITY and nan().
      Add a define for __builtin_ms_va_copy()
      Add tests for the builtin INF and nan() functions.

Oleg Nesterov (3):
      dissect: teach do_expression() to handle EXPR_OFFSETOF
      dissect: teach do_initializer() to handle the nested EXPR_IDENTIFIER's
      dissect: s/mode_t/usage_t/ in report_member()

Omar Sandoval (1):
      sparse-llvm: Fix LLVM 3.5 linker errors

Pavel Roskin (1):
      Use LLVM_CONFIG instead of llvm-config in Makefile

Ramsay Jones (15):
      Add the __restrict__ keyword
      sparse: add 'gnu_inline' to the ignored attributes
      don't call isdigit/tolower with a char argument
      Makefile: suppress error message from shell
      don't run sparse{c,i} tests when sparse-llvm is disabled
      Add support for multiarch system header files
      cgcc: use only the cc command to determine $gcc_base_dir
      cgcc: use $ccom to set $multiarch_dir if not specified
      test-suite: remove bashism to avoid test failures
      cgcc: avoid passing a sparse-only option to cc
      parse.c: remove duplicate 'may_alias' ignored_attributes
      compile-i386.c: don't ignore return value of write(2)
      sparse: add 'alloc_align' to the ignored attributes
      lib: workaround the 'redeclared with different type' errors
      Makefile: pass -Wno-vla to sparse while checking pre-process.c

Randy Dunlap (1):
      documentation: update email reference link

Rui Teng (1):
      sparse: add no_sanitize_address as an ignored attribute

Thomas Graf (1):
      sparse: Make -Werror turn warnigns into errors

Tony Camuso (2):
      .gitignore: add cscope and Qt project files
      Add default case to switches on enum variables

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-12 14:15 Sparse 0.5.1 RC5 released Christopher Li
@ 2017-08-12 20:26 ` Michael Stefaniuc
  2017-08-12 20:51   ` Luc Van Oostenryck
  2017-08-12 21:36 ` Ramsay Jones
  2017-08-13 12:56 ` Dibyendu Majumdar
  2 siblings, 1 reply; 63+ messages in thread
From: Michael Stefaniuc @ 2017-08-12 20:26 UTC (permalink / raw)
  To: Christopher Li; +Cc: Linux-Sparse

On 08/12/2017 04:15 PM, Christopher Li wrote:
> It took longer than I thought. But we are finally there.
> 
> I am very happy that the wine dead loop bug is nailed
> down and the looping while deleting bugs are stopped.
Yepp, thanks. Wine builds fine now.

bye
	michael

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-12 20:26 ` Michael Stefaniuc
@ 2017-08-12 20:51   ` Luc Van Oostenryck
  0 siblings, 0 replies; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-12 20:51 UTC (permalink / raw)
  To: Michael Stefaniuc; +Cc: Christopher Li, Linux-Sparse

On Sat, Aug 12, 2017 at 10:26 PM, Michael Stefaniuc
<mstefani@mykolab.com> wrote:
>> I am very happy that the wine dead loop bug is nailed
>> down and the looping while deleting bugs are stopped.
> Yepp, thanks. Wine builds fine now.

Thanks for the testing!

-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-12 14:15 Sparse 0.5.1 RC5 released Christopher Li
  2017-08-12 20:26 ` Michael Stefaniuc
@ 2017-08-12 21:36 ` Ramsay Jones
  2017-08-12 21:41   ` Christopher Li
  2017-08-13 12:56 ` Dibyendu Majumdar
  2 siblings, 1 reply; 63+ messages in thread
From: Ramsay Jones @ 2017-08-12 21:36 UTC (permalink / raw)
  To: Christopher Li, Linux-Sparse
  Cc: Luc Van Oostenryck, Lance Richardson, Michael Stefaniuc



On 12/08/17 15:15, Christopher Li wrote:
> It took longer than I thought. But we are finally there.
> 
> I am very happy that the wine dead loop bug is nailed
> down and the looping while deleting bugs are stopped.
> 
> Thanks Luc for a lot of hard work there.
> 
> I hope this is the last RC for this release.
> 
> Let's ship this release out. The work for the next release
> is already start to piling up.
> 
> Thanks every one that contribute to this RC5.
> 
> Again, please give it a good test.  Report back
> any issue you might have.

Tested v0.5.1-rc5 on Linux (32- & 64-bit) and (64-bit) cygwin, with
no additional errors or problems. (sparse tests + git).

Thanks!

ATB,
Ramsay Jones


^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-12 21:36 ` Ramsay Jones
@ 2017-08-12 21:41   ` Christopher Li
  0 siblings, 0 replies; 63+ messages in thread
From: Christopher Li @ 2017-08-12 21:41 UTC (permalink / raw)
  To: Ramsay Jones
  Cc: Linux-Sparse, Luc Van Oostenryck, Lance Richardson, Michael Stefaniuc

On Sat, Aug 12, 2017 at 5:36 PM, Ramsay Jones
<ramsay@ramsayjones.plus.com> wrote:
> Tested v0.5.1-rc5 on Linux (32- & 64-bit) and (64-bit) cygwin, with
> no additional errors or problems. (sparse tests + git).
>
Thank you so much for testing.

Chris

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-12 14:15 Sparse 0.5.1 RC5 released Christopher Li
  2017-08-12 20:26 ` Michael Stefaniuc
  2017-08-12 21:36 ` Ramsay Jones
@ 2017-08-13 12:56 ` Dibyendu Majumdar
  2017-08-13 13:10   ` Luc Van Oostenryck
  2017-08-13 19:03   ` Christopher Li
  2 siblings, 2 replies; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-13 12:56 UTC (permalink / raw)
  To: Christopher Li
  Cc: Linux-Sparse, Luc Van Oostenryck, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

Hi Chris & Luc,

On 12 August 2017 at 15:15, Christopher Li <sparse@chrisli.org> wrote:
> It took longer than I thought. But we are finally there.
>
> I am very happy that the wine dead loop bug is nailed
> down and the looping while deleting bugs are stopped.
>
> Thanks Luc for a lot of hard work there.
>
> I hope this is the last RC for this release.
>
> Let's ship this release out. The work for the next release
> is already start to piling up.
>
> Thanks every one that contribute to this RC5.
>
> Again, please give it a good test.  Report back
> any issue you might have.
>

I am merging and testing this in my repository. I notice that after
applying the changes related to killing of BBs - there is a
significant slow down in Sparse.

Before the merge:

real: 4.3 minutes
user: 4.1 minutes

After the merge:

real: 18.9 minutes
user: 18.7 minutes

I just noticed this and these are my initial tests. But I thought it
worth letting you know.

BTW in my tests I do not see any change in the results after the
changes - i.e. the previous failures due to simplification are still
present. But there is now a degradation in performance - so my
question is - are these changes worthwhile?

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 12:56 ` Dibyendu Majumdar
@ 2017-08-13 13:10   ` Luc Van Oostenryck
  2017-08-13 13:18     ` Dibyendu Majumdar
  2017-08-13 19:03   ` Christopher Li
  1 sibling, 1 reply; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-13 13:10 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Sun, Aug 13, 2017 at 2:56 PM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
> Hi Chris & Luc,
>

Hi,

>
> I am merging and testing this in my repository. I notice that after
> applying the changes related to killing of BBs - there is a
> significant slow down in Sparse.
>
> Before the merge:
>
> real: 4.3 minutes
> user: 4.1 minutes
>
> After the merge:
>
> real: 18.9 minutes
> user: 18.7 minutes
>
> I just noticed this and these are my initial tests. But I thought it
> worth letting you know.

I very small slowdown is expected (and even it could have positive
effect too) but not at all something like this.
And what I (and I think Chris too) measured was indeed quite small.

Care to send your code, I can see where the problem could be?

> BTW in my tests I do not see any change in the results after the
> changes - i.e. the previous failures due to simplification are still
> present. But there is now a degradation in performance - so my
> question is - are these changes worthwhile?

This change (well I guess we're talking about the same) is needed
to ensure consistency: avoid to do things in unreachable code where
the conditions to do those things are not met and thus problems follow.

-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 13:10   ` Luc Van Oostenryck
@ 2017-08-13 13:18     ` Dibyendu Majumdar
  2017-08-13 13:54       ` Dibyendu Majumdar
  2017-08-13 14:18       ` Luc Van Oostenryck
  0 siblings, 2 replies; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-13 13:18 UTC (permalink / raw)
  To: Luc Van Oostenryck
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

Hi Luc,

On 13 August 2017 at 14:10, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> On Sun, Aug 13, 2017 at 2:56 PM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>> I am merging and testing this in my repository. I notice that after
>> applying the changes related to killing of BBs - there is a
>> significant slow down in Sparse.
>>
>> Before the merge:
>>
>> real: 4.3 minutes
>> user: 4.1 minutes
>>
>> After the merge:
>>
>> real: 18.9 minutes
>> user: 18.7 minutes
>>
>> I just noticed this and these are my initial tests. But I thought it
>> worth letting you know.
>
> I very small slowdown is expected (and even it could have positive
> effect too) but not at all something like this.
> And what I (and I think Chris too) measured was indeed quite small.
>
> Care to send your code, I can see where the problem could be?
>

Yes apologies I should have mentioned the test case:

https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/lcc/cq.c

I am checking whether the slowdown is due to some other factor such as:

a) Build options - I need to check whether my build is an optimized
build or not.
b) I am using function based ptrlist iterators - so that may be
exacerbating the issue

But still it would be interesting to know what your results are for
this test case.

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 13:18     ` Dibyendu Majumdar
@ 2017-08-13 13:54       ` Dibyendu Majumdar
  2017-08-13 14:24         ` Dibyendu Majumdar
  2017-08-13 14:18       ` Luc Van Oostenryck
  1 sibling, 1 reply; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-13 13:54 UTC (permalink / raw)
  To: Luc Van Oostenryck
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

Hi Luc,

On 13 August 2017 at 14:18, Dibyendu Majumdar <mobile@majumdar.org.uk> wrote:
> On 13 August 2017 at 14:10, Luc Van Oostenryck
> <luc.vanoostenryck@gmail.com> wrote:
>> On Sun, Aug 13, 2017 at 2:56 PM, Dibyendu Majumdar
>> <mobile@majumdar.org.uk> wrote:
>>> I am merging and testing this in my repository. I notice that after
>>> applying the changes related to killing of BBs - there is a
>>> significant slow down in Sparse.
>>>
>>> Before the merge:
>>>
>>> real: 4.3 minutes
>>> user: 4.1 minutes
>>>
>>> After the merge:
>>>
>>> real: 18.9 minutes
>>> user: 18.7 minutes
>>>
>>> I just noticed this and these are my initial tests. But I thought it
>>> worth letting you know.
>>
> Yes apologies I should have mentioned the test case:
>
> https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/lcc/cq.c
>
> I am checking whether the slowdown is due to some other factor such as:
>
> a) Build options - I need to check whether my build is an optimized
> build or not.

I had optimizations switched off - but with -O2, results are not good either:

Before merge:

real: 1.6 minutes
user: 1.5 minutes

After merge:

real: 8.3 minutes
user: 8.2 minutes

I do have function based iterators so that is another thing to check.

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 13:18     ` Dibyendu Majumdar
  2017-08-13 13:54       ` Dibyendu Majumdar
@ 2017-08-13 14:18       ` Luc Van Oostenryck
  2017-08-13 14:25         ` Luc Van Oostenryck
  2017-08-13 14:44         ` Dibyendu Majumdar
  1 sibling, 2 replies; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-13 14:18 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Sun, Aug 13, 2017 at 3:18 PM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:

>> Care to send your code, I can see where the problem could be?
>
> Yes apologies I should have mentioned the test case:
>
> https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/lcc/cq.c

No problem.

> I am checking whether the slowdown is due to some other factor such as:

I just gave a try and, I was a bit surprised as with the current master,
here it takes me about 5s to get the output of test-linearize (which I find
already a bit slow). It's far far from the time you have.

OTOH with -rc4, it takes only 0.05s !

The situation is really weird, it's an interaction between the patches:
- remove single-store shortcut (first slowdown to 0.8s)
- fix ptrlist corruption while killing unreachable BBs (second slowdown to 2.9s)
- mark pseudo users as deleted instead of removing them (third slowdown to 5.5s)

All three patches were expected to create *some* slowdown,
the first one very obviously and much more than the others.

It's interesting to note that without the 'remove single shortcut' patch, the
two others ones don't seem to have a significative effect on the time.

My guess here is that indeed, with this input, the single-store shortcut
is a huge optimization and without it much much more code is created
(phi-nodes, I suppose) and this impact very negatively the two other fixes.

I'll see what can be done.

-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 13:54       ` Dibyendu Majumdar
@ 2017-08-13 14:24         ` Dibyendu Majumdar
  0 siblings, 0 replies; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-13 14:24 UTC (permalink / raw)
  To: Luc Van Oostenryck
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On 13 August 2017 at 14:54, Dibyendu Majumdar <mobile@majumdar.org.uk> wrote:
> On 13 August 2017 at 14:18, Dibyendu Majumdar <mobile@majumdar.org.uk> wrote:
>> On 13 August 2017 at 14:10, Luc Van Oostenryck
>> <luc.vanoostenryck@gmail.com> wrote:
>>> On Sun, Aug 13, 2017 at 2:56 PM, Dibyendu Majumdar
>>> <mobile@majumdar.org.uk> wrote:
>>>> I am merging and testing this in my repository. I notice that after
>>>> applying the changes related to killing of BBs - there is a
>>>> significant slow down in Sparse.
>>>>
>>>> Before the merge:
>>>>
>>>> real: 4.3 minutes
>>>> user: 4.1 minutes
>>>>
>>>> After the merge:
>>>>
>>>> real: 18.9 minutes
>>>> user: 18.7 minutes
>>>>
>>>> I just noticed this and these are my initial tests. But I thought it
>>>> worth letting you know.
>>>
>> Yes apologies I should have mentioned the test case:
>>
>> https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/lcc/cq.c
>>
>> I am checking whether the slowdown is due to some other factor such as:
>>
>> a) Build options - I need to check whether my build is an optimized
>> build or not.
>
> I had optimizations switched off - but with -O2, results are not good either:
>
> Before merge:
>
> real: 1.6 minutes
> user: 1.5 minutes
>
> After merge:
>
> real: 8.3 minutes
> user: 8.2 minutes
>

Apologies all - these times are in seconds - not minutes.

Regards

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 14:18       ` Luc Van Oostenryck
@ 2017-08-13 14:25         ` Luc Van Oostenryck
  2017-08-13 14:26           ` Dibyendu Majumdar
  2017-08-13 16:40           ` Luc Van Oostenryck
  2017-08-13 14:44         ` Dibyendu Majumdar
  1 sibling, 2 replies; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-13 14:25 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

> OTOH with -rc4, it takes only 0.05s !
>
> The situation is really weird, it's an interaction between the patches:
> - remove single-store shortcut (first slowdown to 0.8s)
> - fix ptrlist corruption while killing unreachable BBs (second slowdown to 2.9s)
> - mark pseudo users as deleted instead of removing them (third slowdown to 5.5s)
>
> All three patches were expected to create *some* slowdown,
> the first one very obviously and much more than the others.
>
> It's interesting to note that without the 'remove single shortcut' patch, the
> two others ones don't seem to have a significative effect on the time.
>
> My guess here is that indeed, with this input, the single-store shortcut
> is a huge optimization and without it much much more code is created
> (phi-nodes, I suppose) and this impact very negatively the two other fixes.
>
> I'll see what can be done.

It's also interesting to note that with the new SSA construction, there is no
slowdown (0.064s here but too small to be measured accurately).
This confirms me that the single-store shortcut was very much needed for
performance (not simply because it's faster than the general case but that
it creates much less intermediate code) but it created bad code ...

-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 14:25         ` Luc Van Oostenryck
@ 2017-08-13 14:26           ` Dibyendu Majumdar
  2017-08-13 15:07             ` Luc Van Oostenryck
  2017-08-13 16:40           ` Luc Van Oostenryck
  1 sibling, 1 reply; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-13 14:26 UTC (permalink / raw)
  To: Luc Van Oostenryck
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

Hi Luc,

On 13 August 2017 at 15:25, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
>> OTOH with -rc4, it takes only 0.05s !
>>
>> The situation is really weird, it's an interaction between the patches:
>> - remove single-store shortcut (first slowdown to 0.8s)
>> - fix ptrlist corruption while killing unreachable BBs (second slowdown to 2.9s)
>> - mark pseudo users as deleted instead of removing them (third slowdown to 5.5s)
>>
>> All three patches were expected to create *some* slowdown,
>> the first one very obviously and much more than the others.
>>
>> It's interesting to note that without the 'remove single shortcut' patch, the
>> two others ones don't seem to have a significative effect on the time.
>>
>> My guess here is that indeed, with this input, the single-store shortcut
>> is a huge optimization and without it much much more code is created
>> (phi-nodes, I suppose) and this impact very negatively the two other fixes.
>>
>> I'll see what can be done.
>
> It's also interesting to note that with the new SSA construction, there is no
> slowdown (0.064s here but too small to be measured accurately).
> This confirms me that the single-store shortcut was very much needed for
> performance (not simply because it's faster than the general case but that
> it creates much less intermediate code) but it created bad code ...
>

I wonder if it is worth doing these changes - instead we should
probably look at your new SSA construction approach.

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 14:18       ` Luc Van Oostenryck
  2017-08-13 14:25         ` Luc Van Oostenryck
@ 2017-08-13 14:44         ` Dibyendu Majumdar
  2017-08-13 15:00           ` Dibyendu Majumdar
  1 sibling, 1 reply; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-13 14:44 UTC (permalink / raw)
  To: Luc Van Oostenryck
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On 13 August 2017 at 15:18, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> On Sun, Aug 13, 2017 at 3:18 PM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>
>>> Care to send your code, I can see where the problem could be?
>>
>> Yes apologies I should have mentioned the test case:
>>
>> https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/lcc/cq.c
>
> No problem.
>
>> I am checking whether the slowdown is due to some other factor such as:
>
> I just gave a try and, I was a bit surprised as with the current master,
> here it takes me about 5s to get the output of test-linearize (which I find
> already a bit slow). It's far far from the time you have.
>
> OTOH with -rc4, it takes only 0.05s !
>
> The situation is really weird, it's an interaction between the patches:
> - remove single-store shortcut (first slowdown to 0.8s)
> - fix ptrlist corruption while killing unreachable BBs (second slowdown to 2.9s)
> - mark pseudo users as deleted instead of removing them (third slowdown to 5.5s)
>
> All three patches were expected to create *some* slowdown,
> the first one very obviously and much more than the others.
>
> It's interesting to note that without the 'remove single shortcut' patch, the
> two others ones don't seem to have a significative effect on the time.
>

Yes, reinstating the single store shortcut reduces time from 8.3 secs
to 0.16 seconds.

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 14:44         ` Dibyendu Majumdar
@ 2017-08-13 15:00           ` Dibyendu Majumdar
  2017-08-13 15:13             ` Luc Van Oostenryck
  0 siblings, 1 reply; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-13 15:00 UTC (permalink / raw)
  To: Luc Van Oostenryck
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On 13 August 2017 at 15:44, Dibyendu Majumdar <mobile@majumdar.org.uk> wrote:
> On 13 August 2017 at 15:18, Luc Van Oostenryck
> <luc.vanoostenryck@gmail.com> wrote:
>> On Sun, Aug 13, 2017 at 3:18 PM, Dibyendu Majumdar
>> <mobile@majumdar.org.uk> wrote:
>>
>>>> Care to send your code, I can see where the problem could be?
>>>
>>> Yes apologies I should have mentioned the test case:
>>>
>>> https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/lcc/cq.c
>>
>> No problem.
>>
>>> I am checking whether the slowdown is due to some other factor such as:
>>
>> I just gave a try and, I was a bit surprised as with the current master,
>> here it takes me about 5s to get the output of test-linearize (which I find
>> already a bit slow). It's far far from the time you have.
>>
>> OTOH with -rc4, it takes only 0.05s !
>>
>> The situation is really weird, it's an interaction between the patches:
>> - remove single-store shortcut (first slowdown to 0.8s)
>> - fix ptrlist corruption while killing unreachable BBs (second slowdown to 2.9s)
>> - mark pseudo users as deleted instead of removing them (third slowdown to 5.5s)
>>
>> All three patches were expected to create *some* slowdown,
>> the first one very obviously and much more than the others.
>>
>> It's interesting to note that without the 'remove single shortcut' patch, the
>> two others ones don't seem to have a significative effect on the time.
>>
>
> Yes, reinstating the single store shortcut reduces time from 8.3 secs
> to 0.16 seconds.
>

In the interest of getting the release out I recommend simply undoing
the single store shortcut change. It was a last minute change and that
is always risky.

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 14:26           ` Dibyendu Majumdar
@ 2017-08-13 15:07             ` Luc Van Oostenryck
  2017-08-13 15:12               ` Dibyendu Majumdar
  0 siblings, 1 reply; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-13 15:07 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Sun, Aug 13, 2017 at 4:26 PM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
>>
>> It's also interesting to note that with the new SSA construction, there is no
>> slowdown (0.064s here but too small to be measured accurately).
>> This confirms me that the single-store shortcut was very much needed for
>> performance (not simply because it's faster than the general case but that
>> it creates much less intermediate code) but it created bad code ...
>>
>
> I wonder if it is worth doing these changes - instead we should
> probably look at your new SSA construction approach.

It was of course what I was thinking when I wrote this note.

I've put together the minimal changes for having this on top
of the current master:
    git://github.com/lucvoo/sparse.git sssa-mini

Since yesterday I already gave it a good more testing and
everything good so far.

-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 15:07             ` Luc Van Oostenryck
@ 2017-08-13 15:12               ` Dibyendu Majumdar
  0 siblings, 0 replies; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-13 15:12 UTC (permalink / raw)
  To: Luc Van Oostenryck
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

Hi Luc,

On 13 August 2017 at 16:07, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> On Sun, Aug 13, 2017 at 4:26 PM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>>>
>>> It's also interesting to note that with the new SSA construction, there is no
>>> slowdown (0.064s here but too small to be measured accurately).
>>> This confirms me that the single-store shortcut was very much needed for
>>> performance (not simply because it's faster than the general case but that
>>> it creates much less intermediate code) but it created bad code ...
>>>
>>
>> I wonder if it is worth doing these changes - instead we should
>> probably look at your new SSA construction approach.
>
> It was of course what I was thinking when I wrote this note.
>
> I've put together the minimal changes for having this on top
> of the current master:
>     git://github.com/lucvoo/sparse.git sssa-mini
>
> Since yesterday I already gave it a good more testing and
> everything good so far.
>

That's great. I would recommend getting the current release out by
undoing the single store shortcut change - and then immediately after
that your SSA construction should become the main focus as it will
help avoid wasting time on other issues.

In any case I will give your changes a try in my repository and let
you know if I face any issues.

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 15:00           ` Dibyendu Majumdar
@ 2017-08-13 15:13             ` Luc Van Oostenryck
  0 siblings, 0 replies; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-13 15:13 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Sun, Aug 13, 2017 at 5:00 PM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
>
> In the interest of getting the release out I recommend simply undoing
> the single store shortcut change. It was a last minute change and that
> is always risky.

Thing is that this change was made to solve another problem:
sparse going in infinite loops because this shortcut doesn't
handle uninitialized variables while the general case is correct.
So it's a question of correctness vs. performance.

Of course, I also sent yesterday another, more general, bandaid
for these problems but well ...

-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 14:25         ` Luc Van Oostenryck
  2017-08-13 14:26           ` Dibyendu Majumdar
@ 2017-08-13 16:40           ` Luc Van Oostenryck
  1 sibling, 0 replies; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-13 16:40 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

Another interesting thing is that even without removing the single-store
shortcut when using test-linearize we have an "instruction buffer overflow".
In other words, an instruction (a phi-node for sure) can't be displayed
in a 4096 bytes buffer  (because it as so much phi-sources).

This is something I have already seen before when trying to understand
the origin of some quadratic behaviour I had seen.
In this case it was clearly a problem related to SSA construction which
kinda wrongly accumulated phi-sources from parents.

In the current case we have a phi-node with:
- 241 sources
- 550 VOIDs (sources that have been removed during simplification)
- its BB has a single parent (thus should never have a phi-node).

With the shortcut removed, we then have hundreds and hundreds
of phi-nodes with hundreds of sources which fully exp^lain the
excessive time needed to process them. And it's very much
something possible with the removal of the shortcut since its
goal was to avoid to create phi-nodes when possible.

So this clearly points to the problem of the SSA construction
wich not only create misplaced phi-nodes but also, in some case,
create lots of them while there was no need for them.
The shortcut was very good at hiding this.

-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 12:56 ` Dibyendu Majumdar
  2017-08-13 13:10   ` Luc Van Oostenryck
@ 2017-08-13 19:03   ` Christopher Li
  2017-08-13 19:10     ` Dibyendu Majumdar
  2017-08-14 18:24     ` Christopher Li
  1 sibling, 2 replies; 63+ messages in thread
From: Christopher Li @ 2017-08-13 19:03 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Linux-Sparse, Luc Van Oostenryck, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Sun, Aug 13, 2017 at 8:56 AM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
> Hi Chris & Luc,
>
>
> I am merging and testing this in my repository. I notice that after
> applying the changes related to killing of BBs - there is a
> significant slow down in Sparse.
>
> Before the merge:
>
> real: 4.3 minutes
> user: 4.1 minutes
>
> After the merge:
>
> real: 18.9 minutes
> user: 18.7 minutes
>
> I just noticed this and these are my initial tests. But I thought it

I have done some testing on the rc5 vs rc4 in the kernel compile test.

========rc5=======
1145.45user 514.38system 2:36.93elapsed 1057%CPU (0avgtext+0avgdata
235540maxresident)k
0inputs+12736outputs (0major+127283813minor)pagefaults 0swaps
=================
========rc4=======
1125.58user 523.66system 2:38.09elapsed 1043%CPU (0avgtext+0avgdata
235604maxresident)k
0inputs+12744outputs (0major+126738411minor)pagefaults 0swaps
=================

I don't see the big different there. The second one shows a bit slower
is likely due to I haven't warm up my machine to run the first round.

It is likely that your test C file has a much bigger function than the
average kernel function file.

In for your case, even revert does not help you much right?
Because you want the result IR has more correct SSA.

Chris

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 19:03   ` Christopher Li
@ 2017-08-13 19:10     ` Dibyendu Majumdar
  2017-08-13 19:20       ` Christopher Li
  2017-08-13 20:09       ` Luc Van Oostenryck
  2017-08-14 18:24     ` Christopher Li
  1 sibling, 2 replies; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-13 19:10 UTC (permalink / raw)
  To: Christopher Li
  Cc: Linux-Sparse, Luc Van Oostenryck, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

Hi Chris,

On 13 August 2017 at 20:03, Christopher Li <sparse@chrisli.org> wrote:
> On Sun, Aug 13, 2017 at 8:56 AM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>> Hi Chris & Luc,
>>
>>
>> I am merging and testing this in my repository. I notice that after
>> applying the changes related to killing of BBs - there is a
>> significant slow down in Sparse.
>>
>> Before the merge:
>>
>> real: 4.3 minutes
>> user: 4.1 minutes
>>
>> After the merge:
>>
>> real: 18.9 minutes
>> user: 18.7 minutes
>>
>> I just noticed this and these are my initial tests. But I thought it

Just reminding that these times are in seconds not minutes!

>
> I have done some testing on the rc5 vs rc4 in the kernel compile test.
>
> ========rc5=======
> 1145.45user 514.38system 2:36.93elapsed 1057%CPU (0avgtext+0avgdata
> 235540maxresident)k
> 0inputs+12736outputs (0major+127283813minor)pagefaults 0swaps
> =================
> ========rc4=======
> 1125.58user 523.66system 2:38.09elapsed 1043%CPU (0avgtext+0avgdata
> 235604maxresident)k
> 0inputs+12744outputs (0major+126738411minor)pagefaults 0swaps
> =================
>
> I don't see the big different there. The second one shows a bit slower
> is likely due to I haven't warm up my machine to run the first round.
>
> It is likely that your test C file has a much bigger function than the
> average kernel function file.
>
> In for your case, even revert does not help you much right?
> Because you want the result IR has more correct SSA.
>

I have to disable simplifications anyway to get correct IR in some
cases. With the single store shortcut test that sets bitfield succeeds
even when simplifications are on - but even so, I cannot turn
simplifications on in practice because of other failures.

It is okay by me if you want to leave the change in - I have it so I
can switch it on / off in my project.

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 19:10     ` Dibyendu Majumdar
@ 2017-08-13 19:20       ` Christopher Li
  2017-08-13 19:56         ` Dibyendu Majumdar
  2017-08-13 20:09       ` Luc Van Oostenryck
  1 sibling, 1 reply; 63+ messages in thread
From: Christopher Li @ 2017-08-13 19:20 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Linux-Sparse, Luc Van Oostenryck, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Sun, Aug 13, 2017 at 3:10 PM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
>
> I have to disable simplifications anyway to get correct IR in some
> cases. With the single store shortcut test that sets bitfield succeeds
> even when simplifications are on - but even so, I cannot turn
> simplifications on in practice because of other failures.

So you are saying even the last change does not make
you to turn on the simplification.

> It is okay by me if you want to leave the change in - I have it so I
> can switch it on / off in my project.

Yes, I am still thinking about it. Thank you for letting me know.

Chris

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 19:20       ` Christopher Li
@ 2017-08-13 19:56         ` Dibyendu Majumdar
  0 siblings, 0 replies; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-13 19:56 UTC (permalink / raw)
  To: Christopher Li
  Cc: Linux-Sparse, Luc Van Oostenryck, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

Hi Chris,

On 13 August 2017 at 20:20, Christopher Li <sparse@chrisli.org> wrote:
> On Sun, Aug 13, 2017 at 3:10 PM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>>
>> I have to disable simplifications anyway to get correct IR in some
>> cases. With the single store shortcut test that sets bitfield succeeds
>> even when simplifications are on - but even so, I cannot turn
>> simplifications on in practice because of other failures.
>
> So you are saying even the last change does not make
> you to turn on the simplification.
>

That is correct.

The good news is that I have completed the merge of RC5 into my
project and have run the Sparse tests as well as my own tests. I have
not faced any new issues other than the one above.

With my version of the LLVM backend I am able to compile and run a
fairly large program (after turning off simplifications of course). As
this is a language interpreter - I am able to then run this
interpreter to compile another JIT! You can see the details here:

https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/minilua/run.sh
https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/minilua/dynasm/run.sh

Of course until you are able to merge all the Sparse-LLVM fixes you
will not be able to run these tests, but it shows that we are making
good progress!

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 19:10     ` Dibyendu Majumdar
  2017-08-13 19:20       ` Christopher Li
@ 2017-08-13 20:09       ` Luc Van Oostenryck
  2017-08-13 20:20         ` Dibyendu Majumdar
  1 sibling, 1 reply; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-13 20:09 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Sun, Aug 13, 2017 at 9:10 PM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
>
> I have to disable simplifications anyway to get correct IR in some
> cases.

Care to send some examples you have problems with?

-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 20:09       ` Luc Van Oostenryck
@ 2017-08-13 20:20         ` Dibyendu Majumdar
  2017-08-13 21:16           ` Luc Van Oostenryck
  2017-08-14 16:15           ` Luc Van Oostenryck
  0 siblings, 2 replies; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-13 20:20 UTC (permalink / raw)
  To: Luc Van Oostenryck
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

Hi Luc,

On 13 August 2017 at 21:09, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> On Sun, Aug 13, 2017 at 9:10 PM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>>
>> I have to disable simplifications anyway to get correct IR in some
>> cases.
>
> Care to send some examples you have problems with?
>

Sure - you already know this one:

https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/bugs/simplifybug.c

Additionally following fails if simplifications are turned on. I have
not analysed the reason yet.

https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/minilua/minilua.c

Note also that I have a couple of workarounds in my repository:

http://marc.info/?l=linux-sparse&m=148953605610758&w=2

http://marc.info/?l=linux-sparse&m=148993825216454&w=2

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 20:20         ` Dibyendu Majumdar
@ 2017-08-13 21:16           ` Luc Van Oostenryck
  2017-08-13 22:12             ` Dibyendu Majumdar
  2017-08-14 16:15           ` Luc Van Oostenryck
  1 sibling, 1 reply; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-13 21:16 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Sun, Aug 13, 2017 at 10:20 PM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
>>> I have to disable simplifications anyway to get correct IR in some
>>> cases.
>>
>> Care to send some examples you have problems with?
>>
>
> Sure - you already know this one:
>
> https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/bugs/simplifybug.c

When I'm using all LLVM fixes I've ever submitted and my SSA patches on top
available at:
    git://github.com/lucvoo/sparse.git sssa-next

This simplifybug.c doesn't seem to have problems with sparse-LLVM.
I'm less sure about the generated code, of course.

> Additionally following fails if simplifications are turned on. I have
> not analysed the reason yet.
>
> https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/minilua/minilua.c

This 8000 lines file has a lot of header file dependencies, I'm willing to
take a look at the preprocessed output if you send it.

> Note also that I have a couple of workarounds in my repository:
>
> http://marc.info/?l=linux-sparse&m=148953605610758&w=2

Only half of the function is there. Care to send to full version?

> http://marc.info/?l=linux-sparse&m=148993825216454&w=2

A floating-point test program while sparse has very very poor
fp support ... hmmm and with a header file dependencies.
Care to send the preprocessed output?

Regards,
-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 21:16           ` Luc Van Oostenryck
@ 2017-08-13 22:12             ` Dibyendu Majumdar
  2017-08-13 22:36               ` Luc Van Oostenryck
  2017-08-14  0:30               ` Christopher Li
  0 siblings, 2 replies; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-13 22:12 UTC (permalink / raw)
  To: Luc Van Oostenryck
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On 13 August 2017 at 22:16, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> On Sun, Aug 13, 2017 at 10:20 PM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>> Additionally following fails if simplifications are turned on. I have
>> not analysed the reason yet.
>>
>> https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/minilua/minilua.c
>
> This 8000 lines file has a lot of header file dependencies, I'm willing to
> take a look at the preprocessed output if you send it.
>

Actually after RC5 merge there is a change in behaviour.

Previously it would would fail to compile when simplifications were turned on.
Now:

1) Without single store shortcut it fails to compile. The error generated is:

error: no result for pseudo
minilua.c:5593:24: error: failed to output instruction load.64*
%r13018 <- 16[VOID]

minilua.c:5593:24: error: failed to output load.64*    %r13018 <- 16[VOID]

2) With single store shortcut it appears to compile successfully but
when the executable is run against the 'dynasm' test the test fails.
The un-simplified version works however.

Here are the linearized outputs:

Unsimplified:
https://github.com/dibyendumajumdar/dmr_c/blob/mergesparse/tests/minilua/minilua.lin

Simplified with single store shortcut switched OFF:
https://github.com/dibyendumajumdar/dmr_c/blob/mergesparse/tests/minilua/minilua_opt.lin

Simplified with single store shortcut switched ON:
https://github.com/dibyendumajumdar/dmr_c/blob/mergesparse/tests/minilua/minilua_opt_sss.lin

I am not sure this helps you.

>> Note also that I have a couple of workarounds in my repository:
>>
>> http://marc.info/?l=linux-sparse&m=148953605610758&w=2
>
> Only half of the function is there. Care to send to full version?
>
>> http://marc.info/?l=linux-sparse&m=148993825216454&w=2
>
> A floating-point test program while sparse has very very poor
> fp support ... hmmm and with a header file dependencies.
> Care to send the preprocessed output?
>

Maybe we can look at these ones later ...

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 22:12             ` Dibyendu Majumdar
@ 2017-08-13 22:36               ` Luc Van Oostenryck
  2017-08-13 22:50                 ` Dibyendu Majumdar
  2017-08-13 22:55                 ` Dibyendu Majumdar
  2017-08-14  0:30               ` Christopher Li
  1 sibling, 2 replies; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-13 22:36 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Mon, Aug 14, 2017 at 12:12 AM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:

>
> Actually after RC5 merge there is a change in behaviour.
>
> Previously it would would fail to compile when simplifications were turned on.
> Now:
>
> 1) Without single store shortcut it fails to compile. The error generated is:
>
> error: no result for pseudo
> minilua.c:5593:24: error: failed to output instruction load.64*
> %r13018 <- 16[VOID]
>
> minilua.c:5593:24: error: failed to output load.64*    %r13018 <- 16[VOID]

Yes. Please forget about the single store shortcut: it's broken.
But I'm not sure: do you mean "without the shortcut" or
"without the patch that remove the shortcut"?

> 2) With single store shortcut it appears to compile successfully but
> when the executable is run against the 'dynasm' test the test fails.
> The un-simplified version works however.

But I understood you had tried with the SSA I sent  link.
Have I misunderstood?

> Here are the linearized outputs:
>
> I am not sure this helps you.

Not really, I was expecting the result of the preprocessing to avoid
header dependencies and use the code as you're using it.
For example, sparse doesn't know about the __DMR_C__ macro,
we're not using the same header files, not even the same platform.

>>> Note also that I have a couple of workarounds in my repository:
> ...
> Maybe we can look at these ones later ...

Sure.
But really, having small testcases with a clear description of exactly
what is wrong, help a lot.

Personally, I don't look at the output of sparse-llvm because I don't
find it very interesting and even more so because I know there are
lots of problems already in the IR we feed sparse-llvm with.
But if you send some specific problems, I can for sure at least look at them.

Regards,
-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 22:36               ` Luc Van Oostenryck
@ 2017-08-13 22:50                 ` Dibyendu Majumdar
  2017-08-13 23:03                   ` Luc Van Oostenryck
  2017-08-13 22:55                 ` Dibyendu Majumdar
  1 sibling, 1 reply; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-13 22:50 UTC (permalink / raw)
  To: Luc Van Oostenryck
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On 13 August 2017 at 23:36, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> On Mon, Aug 14, 2017 at 12:12 AM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>
>>
>> Actually after RC5 merge there is a change in behaviour.
>>
>> Previously it would would fail to compile when simplifications were turned on.
>> Now:
>>
>> 1) Without single store shortcut it fails to compile. The error generated is:
>>
>> error: no result for pseudo
>> minilua.c:5593:24: error: failed to output instruction load.64*
>> %r13018 <- 16[VOID]
>>
>> minilua.c:5593:24: error: failed to output load.64*    %r13018 <- 16[VOID]
>
> Yes. Please forget about the single store shortcut: it's broken.
> But I'm not sure: do you mean "without the shortcut" or
> "without the patch that remove the shortcut"?

Sorry this one is with the patch that removed the shortcut.

>
>> 2) With single store shortcut it appears to compile successfully but
>> when the executable is run against the 'dynasm' test the test fails.
>> The un-simplified version works however.
>
> But I understood you had tried with the SSA I sent  link.
> Have I misunderstood?

I haven't incorporated your new SSA yet. This one is with single store
short cut restored.

>
>> Here are the linearized outputs:
>>
>> I am not sure this helps you.
>
> Not really, I was expecting the result of the preprocessing to avoid
> header dependencies and use the code as you're using it.
> For example, sparse doesn't know about the __DMR_C__ macro,
> we're not using the same header files, not even the same platform.

I can save the pre-processed output if that helps. Will send you a
link. The __DMR_C__ macro is just to help me avoid constructs that
Sparse LLVM cannot handle.

> But really, having small testcases with a clear description of exactly
> what is wrong, help a lot.

Well I think that we need a combination of tests - real world
programs, as well as short, to the point, tests. The way I see it - if
we get problems with a real world program, we then start digging and
end up with short test that reproduces the problem. Just creating
simple short tests does not tend to cover all scenarios as it is
impossible to construct a comprehensive test suite (unless we can
reuse someone else's test suite - which is something I am trying to
do).

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 22:36               ` Luc Van Oostenryck
  2017-08-13 22:50                 ` Dibyendu Majumdar
@ 2017-08-13 22:55                 ` Dibyendu Majumdar
  2017-08-13 23:10                   ` Luc Van Oostenryck
  1 sibling, 1 reply; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-13 22:55 UTC (permalink / raw)
  To: Luc Van Oostenryck
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

Hi Luc,

On 13 August 2017 at 23:36, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> Not really, I was expecting the result of the preprocessing to avoid
> header dependencies and use the code as you're using it.
> For example, sparse doesn't know about the __DMR_C__ macro,
> we're not using the same header files, not even the same platform.
>

Here it is:

https://github.com/dibyendumajumdar/dmr_c/blob/mergesparse/tests/minilua/minilua_preprocessed.c

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 22:50                 ` Dibyendu Majumdar
@ 2017-08-13 23:03                   ` Luc Van Oostenryck
  0 siblings, 0 replies; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-13 23:03 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Mon, Aug 14, 2017 at 12:50 AM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
>>> 1) Without single store shortcut it fails to compile. The error generated is:
>>>
>>> error: no result for pseudo
>>> minilua.c:5593:24: error: failed to output instruction load.64*
>>> %r13018 <- 16[VOID]
>>>
>>> minilua.c:5593:24: error: failed to output load.64*    %r13018 <- 16[VOID]
>>
>> Yes. Please forget about the single store shortcut: it's broken.
>> But I'm not sure: do you mean "without the shortcut" or
>> "without the patch that remove the shortcut"?
>
> Sorry this one is with the patch that removed the shortcut.

OK.

>>> Here are the linearized outputs:
>>>
>>> I am not sure this helps you.
>>
>> Not really, I was expecting the result of the preprocessing to avoid
>> header dependencies and use the code as you're using it.
>> For example, sparse doesn't know about the __DMR_C__ macro,
>> we're not using the same header files, not even the same platform.
>
> I can save the pre-processed output if that helps. Will send you a
> link.

Yes, it really helps for lots of reasons.
For example, I wasn't able to reproduce the Wine bug with the plain
file because triggered the problem was in a header file for platform X
version Y and I hadn't any problem here.

> The __DMR_C__ macro is just to help me avoid constructs that
> Sparse LLVM cannot handle.

Yes, I saw that.

>> But really, having small testcases with a clear description of exactly
>> what is wrong, help a lot.
>
> Well I think that we need a combination of tests - real world
> programs, as well as short, to the point, tests. The way I see it - if
> we get problems with a real world program, we then start digging and
> end up with short test that reproduces the problem. Just creating
> simple short tests does not tend to cover all scenarios as it is
> impossible to construct a comprehensive test suite (unless we can
> reuse someone else's test suite - which is something I am trying to
> do).

Sure, I also use some big files for testing too, some very big, but
before you can really begin to look at things you need at least have
an idea of what is wrong and having a small problem save a lot of time.
Yes, finding a small reproducer can be a lot of work.

Regards,
-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 22:55                 ` Dibyendu Majumdar
@ 2017-08-13 23:10                   ` Luc Van Oostenryck
  2017-08-13 23:17                     ` Dibyendu Majumdar
  0 siblings, 1 reply; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-13 23:10 UTC (permalink / raw)
  To: Dibyendu Majumdar; +Cc: Linux-Sparse

On Mon, Aug 14, 2017 at 12:55 AM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
> Here it is:
>
> https://github.com/dibyendumajumdar/dmr_c/blob/mergesparse/tests/minilua/minilua_preprocessed.c

OK, thanks.

I see here:
1) 4 errors with a bitcast wich different sizes on each side.
    I'm a bit surprised as the code seems to do the right thing when
    emitting a bitcast, I'll need to check
3) 3 errors of pointer compare with different types.
    It should be easy to fix.

-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 23:10                   ` Luc Van Oostenryck
@ 2017-08-13 23:17                     ` Dibyendu Majumdar
  2017-08-13 23:20                       ` Dibyendu Majumdar
  2017-08-13 23:22                       ` Luc Van Oostenryck
  0 siblings, 2 replies; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-13 23:17 UTC (permalink / raw)
  To: Luc Van Oostenryck; +Cc: Linux-Sparse

Hi Luc,

On 14 August 2017 at 00:10, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> On Mon, Aug 14, 2017 at 12:55 AM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>> Here it is:
>>
>> https://github.com/dibyendumajumdar/dmr_c/blob/mergesparse/tests/minilua/minilua_preprocessed.c
>
> OK, thanks.
>
> I see here:
> 1) 4 errors with a bitcast wich different sizes on each side.
>     I'm a bit surprised as the code seems to do the right thing when
>     emitting a bitcast, I'll need to check
> 3) 3 errors of pointer compare with different types.
>     It should be easy to fix.
>

When you say you are seeing errors ... do you mean by running
sparse-llvm? That won't necessarily match what I get.

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 23:17                     ` Dibyendu Majumdar
@ 2017-08-13 23:20                       ` Dibyendu Majumdar
  2017-08-13 23:28                         ` Luc Van Oostenryck
  2017-08-13 23:22                       ` Luc Van Oostenryck
  1 sibling, 1 reply; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-13 23:20 UTC (permalink / raw)
  To: Luc Van Oostenryck; +Cc: Linux-Sparse

On 14 August 2017 at 00:17, Dibyendu Majumdar <mobile@majumdar.org.uk> wrote:
> On 14 August 2017 at 00:10, Luc Van Oostenryck
> <luc.vanoostenryck@gmail.com> wrote:
>> On Mon, Aug 14, 2017 at 12:55 AM, Dibyendu Majumdar
>> <mobile@majumdar.org.uk> wrote:
>>> Here it is:
>>>
>>> https://github.com/dibyendumajumdar/dmr_c/blob/mergesparse/tests/minilua/minilua_preprocessed.c
>>
>> OK, thanks.
>>
>> I see here:
>> 1) 4 errors with a bitcast wich different sizes on each side.
>>     I'm a bit surprised as the code seems to do the right thing when
>>     emitting a bitcast, I'll need to check
>> 3) 3 errors of pointer compare with different types.
>>     It should be easy to fix.
>>
>
> When you say you are seeing errors ... do you mean by running
> sparse-llvm? That won't necessarily match what I get.
>

I mean to say that I am interested in fixing the Sparse IR generation
- so if these are errors in the IR you spotted, then great (I am
amazed at how quickly you found the errors!).

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 23:17                     ` Dibyendu Majumdar
  2017-08-13 23:20                       ` Dibyendu Majumdar
@ 2017-08-13 23:22                       ` Luc Van Oostenryck
  2017-08-13 23:25                         ` Dibyendu Majumdar
  1 sibling, 1 reply; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-13 23:22 UTC (permalink / raw)
  To: Dibyendu Majumdar; +Cc: Linux-Sparse

On Mon, Aug 14, 2017 at 1:17 AM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
>> I see here:
>> 1) 4 errors with a bitcast wich different sizes on each side.
>>     I'm a bit surprised as the code seems to do the right thing when
>>     emitting a bitcast, I'll need to check
>> 3) 3 errors of pointer compare with different types.
>>     It should be easy to fix.
>>
>
> When you say you are seeing errors ... do you mean by running
> sparse-llvm?

yes:
Invalid bitcast
  %R36121 = bitcast i32 %R361 to i8
Both operands to ICmp instruction are not of the same type!
  %R378 = icmp ne i8* %R3763, %struct.lua_TValue* %R375
Invalid bitcast
  %R10918 = bitcast i32 %R1091 to i8
Invalid bitcast
  %R12116 = bitcast i32 %R1211 to i8
Both operands to ICmp instruction are not of the same type!
  %R1930 = icmp eq { [184 x i8] }* %R1927, { [24 x i8] }* %ARG2
Both operands to ICmp instruction are not of the same type!
  %R2124 = icmp eq i8* %R212221, %struct.lua_TValue* %R2121
Invalid bitcast
  %R32617 = bitcast i32 %R3261 to i8

> That won't necessarily match what I get.

Yes, I realized already but I have to begin with this.
They don't seem to be hard or conceptual bugs.

-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 23:22                       ` Luc Van Oostenryck
@ 2017-08-13 23:25                         ` Dibyendu Majumdar
  2017-08-13 23:35                           ` Luc Van Oostenryck
  0 siblings, 1 reply; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-13 23:25 UTC (permalink / raw)
  To: Luc Van Oostenryck; +Cc: Linux-Sparse

On 14 August 2017 at 00:22, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> On Mon, Aug 14, 2017 at 1:17 AM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>>> I see here:
>>> 1) 4 errors with a bitcast wich different sizes on each side.
>>>     I'm a bit surprised as the code seems to do the right thing when
>>>     emitting a bitcast, I'll need to check
>>> 3) 3 errors of pointer compare with different types.
>>>     It should be easy to fix.
>>>
>>
>> When you say you are seeing errors ... do you mean by running
>> sparse-llvm?
>
> yes:
> Invalid bitcast
>   %R36121 = bitcast i32 %R361 to i8
> Both operands to ICmp instruction are not of the same type!
>   %R378 = icmp ne i8* %R3763, %struct.lua_TValue* %R375
> Invalid bitcast
>   %R10918 = bitcast i32 %R1091 to i8
> Invalid bitcast
>   %R12116 = bitcast i32 %R1211 to i8
> Both operands to ICmp instruction are not of the same type!
>   %R1930 = icmp eq { [184 x i8] }* %R1927, { [24 x i8] }* %ARG2
> Both operands to ICmp instruction are not of the same type!
>   %R2124 = icmp eq i8* %R212221, %struct.lua_TValue* %R2121
> Invalid bitcast
>   %R32617 = bitcast i32 %R3261 to i8
>
>> That won't necessarily match what I get.
>
> Yes, I realized already but I have to begin with this.
> They don't seem to be hard or conceptual bugs.
>

I was wondering if you get the same Sparse IR error I got - i.e.
pseudo that is VOID. You should be able to see this without going to
LLVM.

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 23:20                       ` Dibyendu Majumdar
@ 2017-08-13 23:28                         ` Luc Van Oostenryck
  2017-08-13 23:35                           ` Dibyendu Majumdar
  0 siblings, 1 reply; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-13 23:28 UTC (permalink / raw)
  To: Dibyendu Majumdar; +Cc: Linux-Sparse

On Mon, Aug 14, 2017 at 1:20 AM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
>
> I mean to say that I am interested in fixing the Sparse IR generation
> - so if these are errors in the IR you spotted, then great

Yes, I know.
But I confess that doing this through sparse-llvm is kinda a pita for me :)

> (I am
> amazed at how quickly you found the errors!).

Hey, I have now a small reproducer for one of the problem:
    char a, b;
    void c(void)
    {
        b = a & 1;
    }

and we can solve it and put in the testsuite to avoid future regressions.

Regards,
-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 23:28                         ` Luc Van Oostenryck
@ 2017-08-13 23:35                           ` Dibyendu Majumdar
  2017-08-13 23:40                             ` Luc Van Oostenryck
  0 siblings, 1 reply; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-13 23:35 UTC (permalink / raw)
  To: Luc Van Oostenryck; +Cc: Linux-Sparse

Hi Luc,

On 14 August 2017 at 00:28, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> On Mon, Aug 14, 2017 at 1:20 AM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>>
>> I mean to say that I am interested in fixing the Sparse IR generation
>> - so if these are errors in the IR you spotted, then great
>
> Yes, I know.
> But I confess that doing this through sparse-llvm is kinda a pita for me :)
>
>> (I am
>> amazed at how quickly you found the errors!).
>
> Hey, I have now a small reproducer for one of the problem:
>     char a, b;
>     void c(void)
>     {
>         b = a & 1;
>     }
>
> and we can solve it and put in the testsuite to avoid future regressions.
>

I am not really sure I understand. In dmrC I get following LLVM output
for above:
; ModuleID = 'out.bc'
source_filename = "dmrC"

@a = global i8 0, align 1
@b = global i8 0, align 1

; Function Attrs: argmemonly nounwind
declare void @llvm.memset.p0i8.i32(i8* nocapture writeonly, i8, i32, i32, i1) #0

define void @c() {
L0:
  %R1 = load i8, i8* @a
  %R2 = sext i8 %R1 to i32
  %R3 = and i32 %R2, 1
  %R31 = trunc i32 %R3 to i8
  store i8 %R31, i8* @b
  ret void
}

attributes #0 = { argmemonly nounwind }


What do you get?

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 23:25                         ` Dibyendu Majumdar
@ 2017-08-13 23:35                           ` Luc Van Oostenryck
  2017-08-13 23:42                             ` Dibyendu Majumdar
  0 siblings, 1 reply; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-13 23:35 UTC (permalink / raw)
  To: Dibyendu Majumdar; +Cc: Linux-Sparse

On Mon, Aug 14, 2017 at 1:25 AM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
>
> I was wondering if you get the same Sparse IR error I got - i.e.
> pseudo that is VOID. You should be able to see this without going to
> LLVM.

No but I'm not surprised as I have no idea of what exactly you have for changes

-rc5 crash with an assert because of missing FPCAST but well, master
has none of the needed patches for LLVM.

So I use the link I gave you previously (sssa-next) which has all
LLVM fixes I made, all floating-point related patches I made,
a few more fixes and optimizations and the SSA on top.

I'm pretty pleased by the results so far but I'm not surprised
at all that some more bugs still live in sparse-llvm.

-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 23:35                           ` Dibyendu Majumdar
@ 2017-08-13 23:40                             ` Luc Van Oostenryck
  2017-08-13 23:44                               ` Dibyendu Majumdar
  0 siblings, 1 reply; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-13 23:40 UTC (permalink / raw)
  To: Dibyendu Majumdar; +Cc: Linux-Sparse

On Mon, Aug 14, 2017 at 1:35 AM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
>
> I am not really sure I understand. In dmrC I get following LLVM output
> for above:
> ; ModuleID = 'out.bc'
> source_filename = "dmrC"
>
> @a = global i8 0, align 1
> @b = global i8 0, align 1
>
> ; Function Attrs: argmemonly nounwind
> declare void @llvm.memset.p0i8.i32(i8* nocapture writeonly, i8, i32, i32, i1) #0
>
> define void @c() {
> L0:
>   %R1 = load i8, i8* @a
>   %R2 = sext i8 %R1 to i32
>   %R3 = and i32 %R2, 1
>   %R31 = trunc i32 %R3 to i8
>   store i8 %R31, i8* @b
>   ret void
> }
>
> attributes #0 = { argmemonly nounwind }
>
>
> What do you get?

Nothing at all since I get LLVM error:
Invalid bitcast
  %R31 = bitcast i32 %R3 to i8

I suppose you must have some fixes I haven't or
one of my other fixes created this problem.
I even think I now know which one.

-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 23:35                           ` Luc Van Oostenryck
@ 2017-08-13 23:42                             ` Dibyendu Majumdar
  2017-08-13 23:47                               ` Luc Van Oostenryck
  0 siblings, 1 reply; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-13 23:42 UTC (permalink / raw)
  To: Luc Van Oostenryck; +Cc: Linux-Sparse

Hi Luc,

On 14 August 2017 at 00:35, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> On Mon, Aug 14, 2017 at 1:25 AM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>>
>> I was wondering if you get the same Sparse IR error I got - i.e.
>> pseudo that is VOID. You should be able to see this without going to
>> LLVM.
>
> No but I'm not surprised as I have no idea of what exactly you have for changes
>
> -rc5 crash with an assert because of missing FPCAST but well, master
> has none of the needed patches for LLVM.
>

But you should be able to run test-linearize and get the Sparse IR
output with RC5 - is that also crashing?

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 23:40                             ` Luc Van Oostenryck
@ 2017-08-13 23:44                               ` Dibyendu Majumdar
  0 siblings, 0 replies; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-13 23:44 UTC (permalink / raw)
  To: Luc Van Oostenryck; +Cc: Linux-Sparse

On 14 August 2017 at 00:40, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> On Mon, Aug 14, 2017 at 1:35 AM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>>
>> I am not really sure I understand. In dmrC I get following LLVM output
>> for above:
>> ; ModuleID = 'out.bc'
>> source_filename = "dmrC"
>>
>> @a = global i8 0, align 1
>> @b = global i8 0, align 1
>>
>> ; Function Attrs: argmemonly nounwind
>> declare void @llvm.memset.p0i8.i32(i8* nocapture writeonly, i8, i32, i32, i1) #0
>>
>> define void @c() {
>> L0:
>>   %R1 = load i8, i8* @a
>>   %R2 = sext i8 %R1 to i32
>>   %R3 = and i32 %R2, 1
>>   %R31 = trunc i32 %R3 to i8
>>   store i8 %R31, i8* @b
>>   ret void
>> }
>>
>> attributes #0 = { argmemonly nounwind }
>>
>>
>> What do you get?
>
> Nothing at all since I get LLVM error:
> Invalid bitcast
>   %R31 = bitcast i32 %R3 to i8
>
> I suppose you must have some fixes I haven't or
> one of my other fixes created this problem.
> I even think I now know which one.
>

I have more fixes (I reported all the issues on the mailing list). I
think I have got to the point that I find most issues are now with
Sparse IR not LLVM backend.

Regards

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 23:42                             ` Dibyendu Majumdar
@ 2017-08-13 23:47                               ` Luc Van Oostenryck
  2017-08-13 23:58                                 ` Luc Van Oostenryck
  0 siblings, 1 reply; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-13 23:47 UTC (permalink / raw)
  To: Dibyendu Majumdar; +Cc: Linux-Sparse

On Mon, Aug 14, 2017 at 1:42 AM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
>
> But you should be able to run test-linearize and get the Sparse IR
> output with RC5 - is that also crashing?

No no.
With this tree I haven't yet found a single way to make sparse or
test-linearize crash.

But I need first the patch: 'fix: cast of OP_AND only valid if it's an OP_CAST'
but it will be for another day.

-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 23:47                               ` Luc Van Oostenryck
@ 2017-08-13 23:58                                 ` Luc Van Oostenryck
  2017-08-14  0:30                                   ` Luc Van Oostenryck
  0 siblings, 1 reply; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-13 23:58 UTC (permalink / raw)
  To: Dibyendu Majumdar; +Cc: Linux-Sparse

On Mon, Aug 14, 2017 at 1:47 AM, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:

OK, I now only have the others three errors:
Both operands to ICmp instruction are not of the same type!
  %R369 = icmp ne i8* %R3673, %struct.lua_TValue* %R366
Both operands to ICmp instruction are not of the same type!
  %R1905 = icmp eq { [120 x i8] }* %R1902, { [16 x i8] }* %ARG2
Both operands to ICmp instruction are not of the same type!
  %R2097 = icmp eq i8* %R209521, %struct.lua_TValue* %R2094

I'll look after a reproducer tomorrow.

-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 23:58                                 ` Luc Van Oostenryck
@ 2017-08-14  0:30                                   ` Luc Van Oostenryck
  0 siblings, 0 replies; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-14  0:30 UTC (permalink / raw)
  To: Dibyendu Majumdar; +Cc: Linux-Sparse

I've updated my tree with all the changes:
  git://github.com/lucvoo/sparse.git sssa-next

With this sparse-llvm do fine with your minilua


Have fun,
-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 22:12             ` Dibyendu Majumdar
  2017-08-13 22:36               ` Luc Van Oostenryck
@ 2017-08-14  0:30               ` Christopher Li
  2017-08-14  0:34                 ` Luc Van Oostenryck
  2017-08-14 18:48                 ` Dibyendu Majumdar
  1 sibling, 2 replies; 63+ messages in thread
From: Christopher Li @ 2017-08-14  0:30 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Luc Van Oostenryck, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Sun, Aug 13, 2017 at 6:12 PM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:

> Simplified with single store shortcut switched OFF:
> https://github.com/dibyendumajumdar/dmr_c/blob/mergesparse/tests/minilua/minilua_opt.lin
>

I haven't go through every thing yet. This above output is definitely
*not* RC5 output. It has push instruction. When you integrate
so much new code, it is hard to say where the slow down come
from.

Chris

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-14  0:30               ` Christopher Li
@ 2017-08-14  0:34                 ` Luc Van Oostenryck
  2017-08-14  0:36                   ` Christopher Li
  2017-08-14 18:48                 ` Dibyendu Majumdar
  1 sibling, 1 reply; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-14  0:34 UTC (permalink / raw)
  To: Christopher Li, Linux-Sparse, Dibyendu Majumdar

Chris,

> it is hard to say where the slow down come

I don't know if you haven't seen the other part of the thread
but all the analysis have already be done.

Regards,
-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-14  0:34                 ` Luc Van Oostenryck
@ 2017-08-14  0:36                   ` Christopher Li
  0 siblings, 0 replies; 63+ messages in thread
From: Christopher Li @ 2017-08-14  0:36 UTC (permalink / raw)
  To: Luc Van Oostenryck; +Cc: Linux-Sparse, Dibyendu Majumdar

On Sun, Aug 13, 2017 at 8:34 PM, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:

> I don't know if you haven't seen the other part of the thread
> but all the analysis have already be done.

Great. As I said in my previous email, I haven't go through every thing yet.

Chris

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 20:20         ` Dibyendu Majumdar
  2017-08-13 21:16           ` Luc Van Oostenryck
@ 2017-08-14 16:15           ` Luc Van Oostenryck
  2017-08-14 16:17             ` Dibyendu Majumdar
  1 sibling, 1 reply; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-14 16:15 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Sun, Aug 13, 2017 at 10:20 PM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
>> Care to send some examples you have problems with?
>
> Sure - you already know this one:
>
> https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/bugs/simplifybug.c
>
> https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/minilua/minilua.c
>

OK, I just checked on the paranoia.c and once the global initializations
are removed sparse-llvm has not more problems when using all the
needed fixes. An updated tree can be found as usual at:
        git://github.com/lucvoo/sparse.git sssa-next

-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-14 16:15           ` Luc Van Oostenryck
@ 2017-08-14 16:17             ` Dibyendu Majumdar
  2017-08-14 16:25               ` Luc Van Oostenryck
  0 siblings, 1 reply; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-14 16:17 UTC (permalink / raw)
  To: Luc Van Oostenryck
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

Hi Luc,

On 14 August 2017 at 17:15, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> On Sun, Aug 13, 2017 at 10:20 PM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>>> Care to send some examples you have problems with?
>>
>> Sure - you already know this one:
>>
>> https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/bugs/simplifybug.c
>>
>> https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/minilua/minilua.c
>>
>
> OK, I just checked on the paranoia.c and once the global initializations
> are removed sparse-llvm has not more problems when using all the
> needed fixes. An updated tree can be found as usual at:
>         git://github.com/lucvoo/sparse.git sssa-next
>

Were you able to run the code generated by Sparse-LLVM - or are you
saying that it compiled without errors?

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-14 16:17             ` Dibyendu Majumdar
@ 2017-08-14 16:25               ` Luc Van Oostenryck
  2017-08-14 16:29                 ` Dibyendu Majumdar
  0 siblings, 1 reply; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-14 16:25 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Mon, Aug 14, 2017 at 6:17 PM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
> Hi Luc,
>
> On 14 August 2017 at 17:15, Luc Van Oostenryck
> <luc.vanoostenryck@gmail.com> wrote:
>> On Sun, Aug 13, 2017 at 10:20 PM, Dibyendu Majumdar
>> <mobile@majumdar.org.uk> wrote:
>>>> Care to send some examples you have problems with?
>>>
>>> Sure - you already know this one:
>>>
>>> https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/bugs/simplifybug.c
>>>
>>> https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/minilua/minilua.c
>>>
>>
>> OK, I just checked on the paranoia.c and once the global initializations
>> are removed sparse-llvm has not more problems when using all the
>> needed fixes. An updated tree can be found as usual at:
>>         git://github.com/lucvoo/sparse.git sssa-next
>>
>
> Were you able to run the code generated by Sparse-LLVM - or are you
> saying that it compiled without errors?

The former. More specifically that you can generate the LLVM bytecode
for the whole files and then see the IR with llvm-dis.

-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-14 16:25               ` Luc Van Oostenryck
@ 2017-08-14 16:29                 ` Dibyendu Majumdar
  2017-08-14 16:44                   ` Luc Van Oostenryck
  0 siblings, 1 reply; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-14 16:29 UTC (permalink / raw)
  To: Luc Van Oostenryck
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On 14 August 2017 at 17:25, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> On Mon, Aug 14, 2017 at 6:17 PM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>> On 14 August 2017 at 17:15, Luc Van Oostenryck
>> <luc.vanoostenryck@gmail.com> wrote:
>>> On Sun, Aug 13, 2017 at 10:20 PM, Dibyendu Majumdar
>>> <mobile@majumdar.org.uk> wrote:
>>>>> Care to send some examples you have problems with?
>>>>
>>>> Sure - you already know this one:
>>>>
>>>> https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/bugs/simplifybug.c
>>>>
>>>> https://github.com/dibyendumajumdar/dmr_c/blob/master/tests/minilua/minilua.c
>>>>
>>>
>>> OK, I just checked on the paranoia.c and once the global initializations
>>> are removed sparse-llvm has not more problems when using all the
>>> needed fixes. An updated tree can be found as usual at:
>>>         git://github.com/lucvoo/sparse.git sssa-next
>>>
>>
>> Were you able to run the code generated by Sparse-LLVM - or are you
>> saying that it compiled without errors?
>
> The former. More specifically that you can generate the LLVM bytecode
> for the whole files and then see the IR with llvm-dis.
>

Okay thanks. I already have a working Sparse-LLVM and have more fixes
than are in your tree. So my interest is really in getting the Sparse
linearized code correct as that is where I now find all the issues
are.

I will look at merging your sssa-mini patches to dmrC. I had a quick
look - presumably the ptrmap ought to be hash table? Anyway I have to
make changes to avoid global state - in dmrC allocators are not
global.

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-14 16:29                 ` Dibyendu Majumdar
@ 2017-08-14 16:44                   ` Luc Van Oostenryck
  2017-08-14 16:54                     ` Dibyendu Majumdar
  0 siblings, 1 reply; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-14 16:44 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Mon, Aug 14, 2017 at 6:29 PM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:

> I will look at merging your sssa-mini patches to dmrC. I had a quick
> look - presumably the ptrmap ought to be hash table?

Yes, it's one of the things plan to do but it's not much important
as it works already quite well in most situations.

This evening, I'll make the change to initialize floats and then it
will cover already a lot of code.
I'm also working to redo everything related to casts, will probably
be finished this evening and should simplify a lots of things,
especially on LLVM's side.

-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-14 16:44                   ` Luc Van Oostenryck
@ 2017-08-14 16:54                     ` Dibyendu Majumdar
  2017-08-14 18:51                       ` Luc Van Oostenryck
  0 siblings, 1 reply; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-14 16:54 UTC (permalink / raw)
  To: Luc Van Oostenryck
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

Hi Luc,

On 14 August 2017 at 17:44, Luc Van Oostenryck
<luc.vanoostenryck@gmail.com> wrote:
> On Mon, Aug 14, 2017 at 6:29 PM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>
>> I will look at merging your sssa-mini patches to dmrC. I had a quick
>> look - presumably the ptrmap ought to be hash table?
>
> Yes, it's one of the things plan to do but it's not much important
> as it works already quite well in most situations.
>
> This evening, I'll make the change to initialize floats and then it
> will cover already a lot of code.
> I'm also working to redo everything related to casts, will probably
> be finished this evening and should simplify a lots of things,
> especially on LLVM's side.
>

I would rather just have the SSA construction bits first, without too
many other changes. In particular I have workarounds for floating
point (I just don't allow simplification for ops involving floating
point values) - and I am keen not to have rework things on the LLVM
side, or have to deal with new opcodes etc.

Regards
Dibyendu

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-13 19:03   ` Christopher Li
  2017-08-13 19:10     ` Dibyendu Majumdar
@ 2017-08-14 18:24     ` Christopher Li
  1 sibling, 0 replies; 63+ messages in thread
From: Christopher Li @ 2017-08-14 18:24 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Linux-Sparse, Luc Van Oostenryck, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Sun, Aug 13, 2017 at 3:03 PM, Christopher Li <sparse@chrisli.org> wrote:
> ========rc5=======
> 1145.45user 514.38system 2:36.93elapsed 1057%CPU (0avgtext+0avgdata
> 235540maxresident)k
> 0inputs+12736outputs (0major+127283813minor)pagefaults 0swaps
> =================
> ========rc4=======
> 1125.58user 523.66system 2:38.09elapsed 1043%CPU (0avgtext+0avgdata
> 235604maxresident)k
> 0inputs+12744outputs (0major+126738411minor)pagefaults 0swaps
> =================
>
BTW,

I bought a new compile server for my testing for sparse. I finally get it set up
and start to crush some numbers. It is doing the sparse kernel checking with
24 threads:

1133.13user 431.11system 1:13.51elapsed 2127%CPU (0avgtext+0avgdata
235616maxresident)k
                                              ^^^^^^^^^^^^^^^^^^^^^^
0inputs+12744outputs (0major+125472993minor)pagefaults 0swaps
1134.75user 431.77system 1:13.55elapsed 2129%CPU (0avgtext+0avgdata
235620maxresident)k
                                            ^^^^^^^^^^^^^^^^^^^^^^^^
0inputs+12736outputs (0major+125473516minor)pagefaults 0swaps
1133.19user 432.63system 1:13.77elapsed 2122%CPU (0avgtext+0avgdata
235660maxresident)k
                                             ^^^^^^^^^^^^^^^^^
0inputs+12736outputs (0major+125473542minor)pagefaults 0swaps

So the overall compile speed is about doubled. And the run to run variance is
very small. Within a second. In the last round I am running some monitor
program to see the server work load so that account for some delay there.

The new server is *very* loud. Other than that, I am very happen about it.

Chris

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-14  0:30               ` Christopher Li
  2017-08-14  0:34                 ` Luc Van Oostenryck
@ 2017-08-14 18:48                 ` Dibyendu Majumdar
  2017-08-14 18:52                   ` Christopher Li
  1 sibling, 1 reply; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-14 18:48 UTC (permalink / raw)
  To: Christopher Li
  Cc: Luc Van Oostenryck, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

Hi Chris,

On 14 August 2017 at 01:30, Christopher Li <sparse@chrisli.org> wrote:
> On Sun, Aug 13, 2017 at 6:12 PM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>
>> Simplified with single store shortcut switched OFF:
>> https://github.com/dibyendumajumdar/dmr_c/blob/mergesparse/tests/minilua/minilua_opt.lin
>>
>
> I haven't go through every thing yet. This above output is definitely
> *not* RC5 output. It has push instruction. When you integrate
> so much new code, it is hard to say where the slow down come
> from.
>

In my tests the removal of single store shortcut makes the time
increase from .2 to 8.2 seconds.
I am compiling following:

https://github.com/dibyendumajumdar/dmr_c/blob/mergesparse/tests/lcc/cq.c


Regards

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-14 16:54                     ` Dibyendu Majumdar
@ 2017-08-14 18:51                       ` Luc Van Oostenryck
  0 siblings, 0 replies; 63+ messages in thread
From: Luc Van Oostenryck @ 2017-08-14 18:51 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Christopher Li, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Mon, Aug 14, 2017 at 6:54 PM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
>
> I would rather just have the SSA construction bits first, without too
> many other changes.

I know and it's why I made the sssa-mini but it contains just that:
only the minimal changes needed for the SSA construction on top
of the current master.

But at least you know what can be done with these fixes and
where you can find them.

-- Luc

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-14 18:48                 ` Dibyendu Majumdar
@ 2017-08-14 18:52                   ` Christopher Li
  2017-08-14 18:56                     ` Christopher Li
  2017-08-14 18:58                     ` Dibyendu Majumdar
  0 siblings, 2 replies; 63+ messages in thread
From: Christopher Li @ 2017-08-14 18:52 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Luc Van Oostenryck, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Mon, Aug 14, 2017 at 2:48 PM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:

>
> https://github.com/dibyendumajumdar/dmr_c/blob/mergesparse/tests/lcc/cq.c
>

Can you run a preprocessor output and put it some where in online?

You can do "sparse -E ...." the rest of normal flags.

That way I can duplicate it here with RC5, otherwise
I need to duplicate your project compile environment.

Chris

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-14 18:52                   ` Christopher Li
@ 2017-08-14 18:56                     ` Christopher Li
  2017-08-14 18:58                     ` Dibyendu Majumdar
  1 sibling, 0 replies; 63+ messages in thread
From: Christopher Li @ 2017-08-14 18:56 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Luc Van Oostenryck, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Mon, Aug 14, 2017 at 2:52 PM, Christopher Li <sparse@chrisli.org> wrote:
>>
>> https://github.com/dibyendumajumdar/dmr_c/blob/mergesparse/tests/lcc/cq.c
>>
>
> Can you run a preprocessor output and put it some where in online?
>
> You can do "sparse -E ...." the rest of normal flags.

Actually, I take that back. I can just use sparse to check that file
without major problems. If saw your #if then I assume it need a lot of
header files.

Chris

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-14 18:52                   ` Christopher Li
  2017-08-14 18:56                     ` Christopher Li
@ 2017-08-14 18:58                     ` Dibyendu Majumdar
  2017-08-14 19:23                       ` Christopher Li
  1 sibling, 1 reply; 63+ messages in thread
From: Dibyendu Majumdar @ 2017-08-14 18:58 UTC (permalink / raw)
  To: Christopher Li
  Cc: Luc Van Oostenryck, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On 14 August 2017 at 19:52, Christopher Li <sparse@chrisli.org> wrote:
> On Mon, Aug 14, 2017 at 2:48 PM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>
>>
>> https://github.com/dibyendumajumdar/dmr_c/blob/mergesparse/tests/lcc/cq.c
>>
>
> Can you run a preprocessor output and put it some where in online?
>
> You can do "sparse -E ...." the rest of normal flags.
>
> That way I can duplicate it here with RC5, otherwise
> I need to duplicate your project compile environment.
>

Here it is:

https://github.com/dibyendumajumdar/dmr_c/blob/mergesparse/tests/lcc/cq_preprocessed.c

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-14 18:58                     ` Dibyendu Majumdar
@ 2017-08-14 19:23                       ` Christopher Li
  2017-08-14 19:47                         ` Christopher Li
  0 siblings, 1 reply; 63+ messages in thread
From: Christopher Li @ 2017-08-14 19:23 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Luc Van Oostenryck, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Mon, Aug 14, 2017 at 2:58 PM, Dibyendu Majumdar
<mobile@majumdar.org.uk> wrote:
>
> Here it is:
>
> https://github.com/dibyendumajumdar/dmr_c/blob/mergesparse/tests/lcc/cq_preprocessed.c

Thanks. Taking a look right now.

Chris

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-14 19:23                       ` Christopher Li
@ 2017-08-14 19:47                         ` Christopher Li
  2017-08-14 20:30                           ` Christopher Li
  0 siblings, 1 reply; 63+ messages in thread
From: Christopher Li @ 2017-08-14 19:47 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Luc Van Oostenryck, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Mon, Aug 14, 2017 at 3:23 PM, Christopher Li <sparse@chrisli.org> wrote:
> On Mon, Aug 14, 2017 at 2:58 PM, Dibyendu Majumdar
> <mobile@majumdar.org.uk> wrote:
>>
>> Here it is:
>>
>> https://github.com/dibyendumajumdar/dmr_c/blob/mergesparse/tests/lcc/cq_preprocessed.c

Most of the time is spending on the s714 function.

That function has a lot of variable name and a lot of blocks.

Chris

^ permalink raw reply	[flat|nested] 63+ messages in thread

* Re: Sparse 0.5.1 RC5 released.
  2017-08-14 19:47                         ` Christopher Li
@ 2017-08-14 20:30                           ` Christopher Li
  0 siblings, 0 replies; 63+ messages in thread
From: Christopher Li @ 2017-08-14 20:30 UTC (permalink / raw)
  To: Dibyendu Majumdar
  Cc: Luc Van Oostenryck, Linux-Sparse, Ramsay Jones, Lance Richardson,
	Michael Stefaniuc

On Mon, Aug 14, 2017 at 3:47 PM, Christopher Li <sparse@chrisli.org> wrote:
>
> Most of the time is spending on the s714 function.
>
> That function has a lot of variable name and a lot of blocks.
>

I think that is the normal path of finding dominating stores
for the  "simplify_loads()"

Chris

^ permalink raw reply	[flat|nested] 63+ messages in thread

end of thread, other threads:[~2017-08-14 20:30 UTC | newest]

Thread overview: 63+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-08-12 14:15 Sparse 0.5.1 RC5 released Christopher Li
2017-08-12 20:26 ` Michael Stefaniuc
2017-08-12 20:51   ` Luc Van Oostenryck
2017-08-12 21:36 ` Ramsay Jones
2017-08-12 21:41   ` Christopher Li
2017-08-13 12:56 ` Dibyendu Majumdar
2017-08-13 13:10   ` Luc Van Oostenryck
2017-08-13 13:18     ` Dibyendu Majumdar
2017-08-13 13:54       ` Dibyendu Majumdar
2017-08-13 14:24         ` Dibyendu Majumdar
2017-08-13 14:18       ` Luc Van Oostenryck
2017-08-13 14:25         ` Luc Van Oostenryck
2017-08-13 14:26           ` Dibyendu Majumdar
2017-08-13 15:07             ` Luc Van Oostenryck
2017-08-13 15:12               ` Dibyendu Majumdar
2017-08-13 16:40           ` Luc Van Oostenryck
2017-08-13 14:44         ` Dibyendu Majumdar
2017-08-13 15:00           ` Dibyendu Majumdar
2017-08-13 15:13             ` Luc Van Oostenryck
2017-08-13 19:03   ` Christopher Li
2017-08-13 19:10     ` Dibyendu Majumdar
2017-08-13 19:20       ` Christopher Li
2017-08-13 19:56         ` Dibyendu Majumdar
2017-08-13 20:09       ` Luc Van Oostenryck
2017-08-13 20:20         ` Dibyendu Majumdar
2017-08-13 21:16           ` Luc Van Oostenryck
2017-08-13 22:12             ` Dibyendu Majumdar
2017-08-13 22:36               ` Luc Van Oostenryck
2017-08-13 22:50                 ` Dibyendu Majumdar
2017-08-13 23:03                   ` Luc Van Oostenryck
2017-08-13 22:55                 ` Dibyendu Majumdar
2017-08-13 23:10                   ` Luc Van Oostenryck
2017-08-13 23:17                     ` Dibyendu Majumdar
2017-08-13 23:20                       ` Dibyendu Majumdar
2017-08-13 23:28                         ` Luc Van Oostenryck
2017-08-13 23:35                           ` Dibyendu Majumdar
2017-08-13 23:40                             ` Luc Van Oostenryck
2017-08-13 23:44                               ` Dibyendu Majumdar
2017-08-13 23:22                       ` Luc Van Oostenryck
2017-08-13 23:25                         ` Dibyendu Majumdar
2017-08-13 23:35                           ` Luc Van Oostenryck
2017-08-13 23:42                             ` Dibyendu Majumdar
2017-08-13 23:47                               ` Luc Van Oostenryck
2017-08-13 23:58                                 ` Luc Van Oostenryck
2017-08-14  0:30                                   ` Luc Van Oostenryck
2017-08-14  0:30               ` Christopher Li
2017-08-14  0:34                 ` Luc Van Oostenryck
2017-08-14  0:36                   ` Christopher Li
2017-08-14 18:48                 ` Dibyendu Majumdar
2017-08-14 18:52                   ` Christopher Li
2017-08-14 18:56                     ` Christopher Li
2017-08-14 18:58                     ` Dibyendu Majumdar
2017-08-14 19:23                       ` Christopher Li
2017-08-14 19:47                         ` Christopher Li
2017-08-14 20:30                           ` Christopher Li
2017-08-14 16:15           ` Luc Van Oostenryck
2017-08-14 16:17             ` Dibyendu Majumdar
2017-08-14 16:25               ` Luc Van Oostenryck
2017-08-14 16:29                 ` Dibyendu Majumdar
2017-08-14 16:44                   ` Luc Van Oostenryck
2017-08-14 16:54                     ` Dibyendu Majumdar
2017-08-14 18:51                       ` Luc Van Oostenryck
2017-08-14 18:24     ` Christopher Li

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.