How does BTRFS compression influence existing and new snapshots?

How does BTRFS compression influence existing and new snapshots?

[Thorsten Schöning]

Hi everyone...

# Context

I have some Synology NAS storing backups created by WBADMIN from
Windows using SMB, which means VHDX image files in the end with
compression disabled, as otherwise WBADMIN refuses to work on those
files. Those files need to still be available on the NAS for some
time, but it's VERY unlikely that those need to be mounted again by
anyone, therefore I would like to apply compression afterwards now to
simply safe some space. If it's ever need to be mounted again, things
can easily be decompressed, so let's ignore that for now. DSM provides
all the necessary tools[1] to deal with compression of existing data
on the shell as well:         

> chattr -R +c [...]
> btrfs filesystem defragment -r -c [...]

The important thing to note is that the existing VHDX files are
protected by automatically created snapshots already and the NAS
continues to create snapshots automatically after I compressed the
files. So what happens with those snapshots?

# Are blocks hold by existing snapshots compressed as well?

According this[2] explanation I don't think so. Additionally this
would somewhat defeat the whole purpose of snapshots guaranteeing
unchanged data for some point in time. Things simply were not
compressed in the past when the snapshots were created, so in theory
that data needs to be available somehow.

OTOH, compression is designed to be somewhat transparent anyway
already and that might be argued all the way through to existing
blocks even in snapshots. Might safe a lot of space in the end.
Compression even is that transparent that "du"[3] is not able to
recognize[4].

# Are newly compressed blocks changes hold by new snapshots?

I have a snapshot BEFORE compressing, compress the VHDX files and
create a new snapshot AFTERWARDS. Nothing else changes in the VHDX by
Windows or WBADMIN or whomever, so from a logical point of view the
file is still unchanged. Though, the individual blocks/extents of the
file managed by BTRFS have been changed a lot, depending on how good
compression has been applied.

This results in actual storage newly allocated between the two
snapshots, doesn't it? So after compression, until earlier snapshots
holding uncompressed data are deleted, overall storage might simply be
less than before.

Or am I wrong somewhere? Thanks!

[1]: https://community.synology.com/enu/forum/1/post/138784
[2]: https://superuser.com/a/892293/308859
[3]: https://stackoverflow.com/a/20899536/2055163
[4]: https://btrfs.wiki.kernel.org/index.php/Compression#Why_does_not_du_report_the_compressed_size.3F

Mit freundlichen Grüßen

Thorsten Schöning

-- 
AM-SoFT IT-Service - Bitstore Hameln GmbH i.G.
Mitglied der Bitstore Gruppe - Ihr Full-Service-Dienstleister für IT und TK

E-Mail: Thorsten.Schoening@AM-SoFT.de
Web:    http://www.AM-SoFT.de/

Tel:   05151-  9468- 0
Tel:   05151-  9468-55
Fax:   05151-  9468-88
Mobil:  0178-8 9468-04

AM-SoFT IT-Service - Bitstore Hameln GmbH i.G., Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB neu - Geschäftsführer: Janine Galonska


Für Rückfragen stehe ich Ihnen sehr gerne zur Verfügung.

Mit freundlichen Grüßen

Thorsten Schöning


Tel: 05151 9468 0
Fax: 05151 9468 88
Mobil: 
Webseite: https://www.am-soft.de 

AM-Soft IT-Service - Bitstore Hameln GmbH i.G. ist ein Mitglied der Bitstore Gruppe - Ihr Full-Service-Dienstleister für IT und TK

AM-Soft IT-Service - Bitstore Hameln GmbH i.G.
Brandenburger Str. 7c
31789 Hameln
Tel: 05151 9468 0

Bitstore IT-Consulting GmbH
Zentrale - Berlin Lichtenberg
Frankfurter Allee 285
10317 Berlin
Tel: 030 453 087 80

CBS IT-Service - Bitstore Kaulsdorf UG
Tel: 030 453 087 880 1

Büro Dallgow-Döberitz
Tel: 03322 507 020

Büro Kloster Lehnin
Tel: 033207 566 530

PCE IT-Service - Bitstore Darmstadt UG
Darmstadt
Tel: 06151 392 973 0

Büro Neuruppin
Tel: 033932 606 090

ACI EDV Systemhaus - Bitstore Dresden GmbH
Dresden
Tel: 0351 254 410

Das Systemhaus - Bitstore Magdeburg GmbH
Magdeburg
Tel: 0391 636 651 0

Allerdata.IT - Bitstore Wittenberg GmbH
Wittenberg
Tel: 03491 876 735 7

Büro Liebenwalde
Tel: 033054 810 00

HSA - das Büro - Bitstore Altenburg UG
Altenburg
Tel: 0344 784 390 97

Bitstore IT – Consulting GmbH
NL Piesteritz 
Piesteritz
Tel: 03491 644 868 6

Solltec IT-Services - Bitstore Braunschweig UG
Braunschweig
Tel: 0531 206 068 0

MF Computer Service - Bitstore Gütersloh GmbH
Gütersloh
Tel: 05245 920 809 3

Firmensitz: AM-Soft IT-Service - Bitstore Hameln GmbH i.G. , Brandenburger Str. 7c , 31789 Hameln
Geschäftsführer Janine Galonska

Re: How does BTRFS compression influence existing and new snapshots?

[Zygo Blaxell]

On Fri, May 28, 2021 at 06:02:04PM +0200, Thorsten Schöning wrote:
> Hi everyone...
> 
> # Context
> 
> I have some Synology NAS storing backups created by WBADMIN from
> Windows using SMB, which means VHDX image files in the end with
> compression disabled, as otherwise WBADMIN refuses to work on those
> files. Those files need to still be available on the NAS for some
> time, but it's VERY unlikely that those need to be mounted again by
> anyone, therefore I would like to apply compression afterwards now to
> simply safe some space. If it's ever need to be mounted again, things
> can easily be decompressed, so let's ignore that for now. DSM provides
> all the necessary tools[1] to deal with compression of existing data
> on the shell as well:         
> 
> > chattr -R +c [...]
> > btrfs filesystem defragment -r -c [...]
> 
> The important thing to note is that the existing VHDX files are
> protected by automatically created snapshots already and the NAS
> continues to create snapshots automatically after I compressed the
> files. So what happens with those snapshots?

Nothing happens to the snapshots.  'fi defrag' will make new, possibly
compressed (but possibly not compressed) duplicate copies of the data
in the listed files.  These copies will use separate storage space from
the snapshots.

These commands are more or less equivalent:

	btrfs fi defrag -czstd foo

	touch foo.tmp; setfattr -n btrfs.compression -v zstd foo.tmp;
	cp -a foo foo.tmp; mv -f foo.tmp foo

There are some differences:  Defrag doesn't create a new inode, defrag
protects 'foo' against modification during the defrag process, and
defrag tries slightly harder to allocate contiguous space; however,
in either case, an entirely new copy of the file data is created
alongside the old data.  If the old data has no other references
(i.e. no snapshots) it is removed; otherwise, the old data remains.

> # Are blocks hold by existing snapshots compressed as well?

Compression affects only new writes--never any existing data already
on disk.

Each individual extent written on the filesystem contains its own
independent copy of compression parameters (well, _de_compression
parameters, strictly speaking).  It is possible for a single file
to use all 4 compression algorithms (none, zlib, lzo, and zstd) for
different extents.

> According this[2] explanation I don't think so. Additionally this
> would somewhat defeat the whole purpose of snapshots guaranteeing
> unchanged data for some point in time. Things simply were not
> compressed in the past when the snapshots were created, so in theory
> that data needs to be available somehow.
> 
> OTOH, compression is designed to be somewhat transparent anyway
> already and that might be argued all the way through to existing
> blocks even in snapshots. Might safe a lot of space in the end.
> Compression even is that transparent that "du"[3] is not able to
> recognize[4].
> 
> # Are newly compressed blocks changes hold by new snapshots?
> 
> I have a snapshot BEFORE compressing, compress the VHDX files and
> create a new snapshot AFTERWARDS. Nothing else changes in the VHDX by
> Windows or WBADMIN or whomever, so from a logical point of view the
> file is still unchanged. Though, the individual blocks/extents of the
> file managed by BTRFS have been changed a lot, depending on how good
> compression has been applied.

It doesn't matter whether the compression worked or not.  Defrag normally
makes a new copy of the data, and if there are other references to the
data then they are not disturbed.  In this case will be two complete
copies of the data:  one in the snapshot before, and one in the snapshot
after.

> This results in actual storage newly allocated between the two
> snapshots, doesn't it? So after compression, until earlier snapshots
> holding uncompressed data are deleted, overall storage might simply be
> less than before.

I think you meant overall free space will be less.  If so, then yes,
the compressed data and uncompressed data are separately stored duplicate
copies.

You could dedupe the snapshot using the new (compressed) snapshot as a
data source and the old (uncompressed) snapshot as dedupe destination.
You would have to create a custom tool for this, as none of the existing
dedupers are set up for this use case AFAIK.

> Or am I wrong somewhere? Thanks!
> 
> [1]: https://community.synology.com/enu/forum/1/post/138784
> [2]: https://superuser.com/a/892293/308859
> [3]: https://stackoverflow.com/a/20899536/2055163
> [4]: https://btrfs.wiki.kernel.org/index.php/Compression#Why_does_not_du_report_the_compressed_size.3F
> 
> Mit freundlichen Grüßen
> 
> Thorsten Schöning
> 
> -- 
> AM-SoFT IT-Service - Bitstore Hameln GmbH i.G.
> Mitglied der Bitstore Gruppe - Ihr Full-Service-Dienstleister für IT und TK
> 
> E-Mail: Thorsten.Schoening@AM-SoFT.de
> Web:    http://www.AM-SoFT.de/
> 
> Tel:   05151-  9468- 0
> Tel:   05151-  9468-55
> Fax:   05151-  9468-88
> Mobil:  0178-8 9468-04
> 
> AM-SoFT IT-Service - Bitstore Hameln GmbH i.G., Brandenburger Str. 7c, 31789 Hameln
> AG Hannover HRB neu - Geschäftsführer: Janine Galonska
> 
> 
> Für Rückfragen stehe ich Ihnen sehr gerne zur Verfügung.
> 
> Mit freundlichen Grüßen
> 
> Thorsten Schöning
> 
> 
> Tel: 05151 9468 0
> Fax: 05151 9468 88
> Mobil: 
> Webseite: https://www.am-soft.de 
> 
> AM-Soft IT-Service - Bitstore Hameln GmbH i.G. ist ein Mitglied der Bitstore Gruppe - Ihr Full-Service-Dienstleister für IT und TK
> 
> AM-Soft IT-Service - Bitstore Hameln GmbH i.G.
> Brandenburger Str. 7c
> 31789 Hameln
> Tel: 05151 9468 0
> 
> Bitstore IT-Consulting GmbH
> Zentrale - Berlin Lichtenberg
> Frankfurter Allee 285
> 10317 Berlin
> Tel: 030 453 087 80
> 
> CBS IT-Service - Bitstore Kaulsdorf UG
> Tel: 030 453 087 880 1
> 
> Büro Dallgow-Döberitz
> Tel: 03322 507 020
> 
> Büro Kloster Lehnin
> Tel: 033207 566 530
> 
> PCE IT-Service - Bitstore Darmstadt UG
> Darmstadt
> Tel: 06151 392 973 0
> 
> Büro Neuruppin
> Tel: 033932 606 090
> 
> ACI EDV Systemhaus - Bitstore Dresden GmbH
> Dresden
> Tel: 0351 254 410
> 
> Das Systemhaus - Bitstore Magdeburg GmbH
> Magdeburg
> Tel: 0391 636 651 0
> 
> Allerdata.IT - Bitstore Wittenberg GmbH
> Wittenberg
> Tel: 03491 876 735 7
> 
> Büro Liebenwalde
> Tel: 033054 810 00
> 
> HSA - das Büro - Bitstore Altenburg UG
> Altenburg
> Tel: 0344 784 390 97
> 
> Bitstore IT – Consulting GmbH
> NL Piesteritz 
> Piesteritz
> Tel: 03491 644 868 6
> 
> Solltec IT-Services - Bitstore Braunschweig UG
> Braunschweig
> Tel: 0531 206 068 0
> 
> MF Computer Service - Bitstore Gütersloh GmbH
> Gütersloh
> Tel: 05245 920 809 3
> 
> Firmensitz: AM-Soft IT-Service - Bitstore Hameln GmbH i.G. , Brandenburger Str. 7c , 31789 Hameln
> Geschäftsführer Janine Galonska
> 
> 
> 
> 
> 
> 

Re: How does BTRFS compression influence existing and new snapshots?

[Thorsten Schöning]

Guten Tag Zygo Blaxell,
am Freitag, 28. Mai 2021 um 21:36 schrieben Sie:

> Nothing happens to the snapshots.  'fi defrag' will make new, possibly
> compressed (but possibly not compressed) duplicate copies of the data
> in the listed files.  These copies will use separate storage space from
> the snapshots.

Thanks for clearing things up, that what's I expected already.

> Each individual extent written on the filesystem contains its own
> independent copy of compression parameters[...]

This brings me to the following in the wiki, not sure if it's worth an
additional thread:

> There is a simple decision logic: if the first portion of data being
> compressed is not smaller than the original, the compression of the
> file is disabled[...]

https://btrfs.wiki.kernel.org/index.php/Compression#What_happens_to_incompressible_files.3F

If (de)compression methods are managed by extents always anway, is the
statement in the wiki really true that compression of a whole FILE is
disabled if the first portion can't be compressed? Or does the quoted
sentence refer to EXTENTS instead of whole FILES instead?

"btrfs-compsize" prints the following for one of my directories, which
means that at least some parts of the file are compressed, others are
not.

> Processed 230 files, 754137 regular extents (754137 refs), 3 inline.
> Type       Perc     Disk Usage   Uncompressed Referenced
> TOTAL       87%      163G         187G         187G
> none       100%      130G         130G         130G
> lzo         57%       32G          56G          56G

So does this really mean that I was simply lucky because "the first
portion" of the file could be compressed? If that wouldn't the case,
the whole file would be uncompressed even though other parts of the
file might be compressed pretty well?

Referrring to the FILE in the wiki instead of EXTENTS doesn't make too
much sense to me currently.

Mit freundlichen Grüßen

Thorsten Schöning

-- 
AM-SoFT IT-Service - Bitstore Hameln GmbH i.G.
Mitglied der Bitstore Gruppe - Ihr Full-Service-Dienstleister für IT und TK

E-Mail: Thorsten.Schoening@AM-SoFT.de
Web:    http://www.AM-SoFT.de/

Tel:   05151-  9468- 0
Tel:   05151-  9468-55
Fax:   05151-  9468-88
Mobil:  0178-8 9468-04

AM-SoFT IT-Service - Bitstore Hameln GmbH i.G., Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB neu - Geschäftsführer: Janine Galonska


Für Rückfragen stehe ich Ihnen sehr gerne zur Verfügung.

Mit freundlichen Grüßen

Thorsten Schöning


Tel: 05151 9468 0
Fax: 05151 9468 88
Mobil: 
Webseite: https://www.am-soft.de 

AM-Soft IT-Service - Bitstore Hameln GmbH i.G. ist ein Mitglied der Bitstore Gruppe - Ihr Full-Service-Dienstleister für IT und TK

AM-Soft IT-Service - Bitstore Hameln GmbH i.G.
Brandenburger Str. 7c
31789 Hameln
Tel: 05151 9468 0

Bitstore IT-Consulting GmbH
Zentrale - Berlin Lichtenberg
Frankfurter Allee 285
10317 Berlin
Tel: 030 453 087 80

CBS IT-Service - Bitstore Kaulsdorf UG
Tel: 030 453 087 880 1

Büro Dallgow-Döberitz
Tel: 03322 507 020

Büro Kloster Lehnin
Tel: 033207 566 530

PCE IT-Service - Bitstore Darmstadt UG
Darmstadt
Tel: 06151 392 973 0

Büro Neuruppin
Tel: 033932 606 090

ACI EDV Systemhaus - Bitstore Dresden GmbH
Dresden
Tel: 0351 254 410

Das Systemhaus - Bitstore Magdeburg GmbH
Magdeburg
Tel: 0391 636 651 0

Allerdata.IT - Bitstore Wittenberg GmbH
Wittenberg
Tel: 03491 876 735 7

Büro Liebenwalde
Tel: 033054 810 00

HSA - das Büro - Bitstore Altenburg UG
Altenburg
Tel: 0344 784 390 97

Bitstore IT – Consulting GmbH
NL Piesteritz 
Piesteritz
Tel: 03491 644 868 6

Solltec IT-Services - Bitstore Braunschweig UG
Braunschweig
Tel: 0531 206 068 0

MF Computer Service - Bitstore Gütersloh GmbH
Gütersloh
Tel: 05245 920 809 3

Firmensitz: AM-Soft IT-Service - Bitstore Hameln GmbH i.G. , Brandenburger Str. 7c , 31789 Hameln
Geschäftsführer Janine Galonska

Re: How does BTRFS compression influence existing and new snapshots?

[Zygo Blaxell]

On Sat, May 29, 2021 at 11:53:30AM +0200, Thorsten Schöning wrote:
> Guten Tag Zygo Blaxell,
> am Freitag, 28. Mai 2021 um 21:36 schrieben Sie:
> 
> > Nothing happens to the snapshots.  'fi defrag' will make new, possibly
> > compressed (but possibly not compressed) duplicate copies of the data
> > in the listed files.  These copies will use separate storage space from
> > the snapshots.
> 
> Thanks for clearing things up, that what's I expected already.
> 
> > Each individual extent written on the filesystem contains its own
> > independent copy of compression parameters[...]
> 
> This brings me to the following in the wiki, not sure if it's worth an
> additional thread:
> 
> > There is a simple decision logic: if the first portion of data being
> > compressed is not smaller than the original, the compression of the
> > file is disabled[...]
> 
> https://btrfs.wiki.kernel.org/index.php/Compression#What_happens_to_incompressible_files.3F
> 
> If (de)compression methods are managed by extents always anway, is the
> statement in the wiki really true that compression of a whole FILE is
> disabled if the first portion can't be compressed? Or does the quoted
> sentence refer to EXTENTS instead of whole FILES instead?

A bit of both.  Compression is controlled by file attributes, but the
result of compression is stored in extent attributes.

Here's the logic for handing a file write in current kernels:

	1.  If the file cannot be compressed, do not compress.
	This includes files with nodatacow or nodatasum attributes set.
	Also, compression is permanently disabled after fallocate is
	used on a file.

	2.  If the compress-force mount option is used, go to compress.

	3.  If defrag -c is running, go to compress.

	4.  If the no-compress attribute is set, do not compress.

	5.  If the compress mount option is used, or the file has
	the compress fsattr or btrfs.compression xattr set, run a
	heuristic algorithm to determine whether to compress.  If
	the heuristic algorithm does not recommend compression,
	then do not compress; otherwise, go to compress.

	6.  Otherwise, do not compress.

If we reach a "go to compress" case above, then btrfs attempts to compress
the data.  If the compressed result is smaller than the original (rounded
up to page size for non-inline extents), the compressed data is written
and we are done.

If compression made an equal or larger extent, then btrfs will set the
no-compress attribute for the inode.  Any future write to the file that
reaches step 4 will not be compressed.

defrag -c and compress-force always attempt to compress data if
compression is possible for the file, and never set the no-compress
attribute.

Note that the heuristic at step 5 means that compression is not
necessarily disabled on the first incompressible data to a file.  In most
cases, the heuristic algorithm will write those extents uncompressed
without attempting compression.  To disable compression, the data has
to get past the heuristic algorithm and then fail on the real compression.

Also note the heuristic at step 5 is not entirely accurate, so it will
reduce the overall compression ratio because of false positives (i.e.
it will sometimes reject compressible data).  compress-force will increase
the compression ratio by brute force, but as a side-effect (and possible
bug) it also limits the maximum extent size when compression fails, so
metadata size will increase.

> "btrfs-compsize" prints the following for one of my directories, which
> means that at least some parts of the file are compressed, others are
> not.
> 
> > Processed 230 files, 754137 regular extents (754137 refs), 3 inline.
> > Type       Perc     Disk Usage   Uncompressed Referenced
> > TOTAL       87%      163G         187G         187G
> > none       100%      130G         130G         130G
> > lzo         57%       32G          56G          56G
> 
> So does this really mean that I was simply lucky because "the first
> portion" of the file could be compressed? If that wouldn't the case,
> the whole file would be uncompressed even though other parts of the
> file might be compressed pretty well?

If you use the compress mount option, if you are doing many small writes
to a file, or the file contains a mix of compressed and uncompressed data,
you'll find compression eventually gets disabled, though not necessarily
on the first incompressible write.

If you are running a backup or similar server, you want the smallest size
possible, and you don't care about write latency, you probably want to
mount with compress-force, which overrides the nocompress attribute.

> Referrring to the FILE in the wiki instead of EXTENTS doesn't make too
> much sense to me currently.

Compression always takes place in the context of a file to set the
compression parameters.  Even the extents on disk need a file reference to
determine the compression type for reading.

> Mit freundlichen Grüßen
> 
> Thorsten Schöning
> 
> -- 
> AM-SoFT IT-Service - Bitstore Hameln GmbH i.G.
> Mitglied der Bitstore Gruppe - Ihr Full-Service-Dienstleister für IT und TK
> 
> E-Mail: Thorsten.Schoening@AM-SoFT.de
> Web:    http://www.AM-SoFT.de/
> 
> Tel:   05151-  9468- 0
> Tel:   05151-  9468-55
> Fax:   05151-  9468-88
> Mobil:  0178-8 9468-04
> 
> AM-SoFT IT-Service - Bitstore Hameln GmbH i.G., Brandenburger Str. 7c, 31789 Hameln
> AG Hannover HRB neu - Geschäftsführer: Janine Galonska
> 
> 
> Für Rückfragen stehe ich Ihnen sehr gerne zur Verfügung.
> 
> Mit freundlichen Grüßen
> 
> Thorsten Schöning
> 
> 
> Tel: 05151 9468 0
> Fax: 05151 9468 88
> Mobil: 
> Webseite: https://www.am-soft.de 
> 
> AM-Soft IT-Service - Bitstore Hameln GmbH i.G. ist ein Mitglied der Bitstore Gruppe - Ihr Full-Service-Dienstleister für IT und TK
> 
> AM-Soft IT-Service - Bitstore Hameln GmbH i.G.
> Brandenburger Str. 7c
> 31789 Hameln
> Tel: 05151 9468 0
> 
> Bitstore IT-Consulting GmbH
> Zentrale - Berlin Lichtenberg
> Frankfurter Allee 285
> 10317 Berlin
> Tel: 030 453 087 80
> 
> CBS IT-Service - Bitstore Kaulsdorf UG
> Tel: 030 453 087 880 1
> 
> Büro Dallgow-Döberitz
> Tel: 03322 507 020
> 
> Büro Kloster Lehnin
> Tel: 033207 566 530
> 
> PCE IT-Service - Bitstore Darmstadt UG
> Darmstadt
> Tel: 06151 392 973 0
> 
> Büro Neuruppin
> Tel: 033932 606 090
> 
> ACI EDV Systemhaus - Bitstore Dresden GmbH
> Dresden
> Tel: 0351 254 410
> 
> Das Systemhaus - Bitstore Magdeburg GmbH
> Magdeburg
> Tel: 0391 636 651 0
> 
> Allerdata.IT - Bitstore Wittenberg GmbH
> Wittenberg
> Tel: 03491 876 735 7
> 
> Büro Liebenwalde
> Tel: 033054 810 00
> 
> HSA - das Büro - Bitstore Altenburg UG
> Altenburg
> Tel: 0344 784 390 97
> 
> Bitstore IT – Consulting GmbH
> NL Piesteritz 
> Piesteritz
> Tel: 03491 644 868 6
> 
> Solltec IT-Services - Bitstore Braunschweig UG
> Braunschweig
> Tel: 0531 206 068 0
> 
> MF Computer Service - Bitstore Gütersloh GmbH
> Gütersloh
> Tel: 05245 920 809 3
> 
> Firmensitz: AM-Soft IT-Service - Bitstore Hameln GmbH i.G. , Brandenburger Str. 7c , 31789 Hameln
> Geschäftsführer Janine Galonska
> 
> 
> 
> 
> 
> 

Re: How does BTRFS compression influence existing and new snapshots?

[Thorsten Schöning]

Guten Tag Zygo Blaxell,
am Sonntag, 30. Mai 2021 um 00:16 schrieben Sie:

> If compression made an equal or larger extent, then btrfs will set the
> no-compress attribute for the inode.  Any future write to the file that
> reaches step 4 will not be compressed.

Thanks again, your answer might be a good enhancement for the wiki.

Mit freundlichen Grüßen

Thorsten Schöning

-- 
AM-SoFT IT-Service - Bitstore Hameln GmbH i.G.
Mitglied der Bitstore Gruppe - Ihr Full-Service-Dienstleister für IT und TK

E-Mail: Thorsten.Schoening@AM-SoFT.de
Web:    http://www.AM-SoFT.de/

Tel:   05151-  9468- 0
Tel:   05151-  9468-55
Fax:   05151-  9468-88
Mobil:  0178-8 9468-04

AM-SoFT IT-Service - Bitstore Hameln GmbH i.G., Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB neu - Geschäftsführer: Janine Galonska


Für Rückfragen stehe ich Ihnen sehr gerne zur Verfügung.

Mit freundlichen Grüßen

Thorsten Schöning


Tel: 05151 9468 0
Fax: 05151 9468 88
Mobil: 
Webseite: https://www.am-soft.de 

AM-Soft IT-Service - Bitstore Hameln GmbH i.G. ist ein Mitglied der Bitstore Gruppe - Ihr Full-Service-Dienstleister für IT und TK

AM-Soft IT-Service - Bitstore Hameln GmbH i.G.
Brandenburger Str. 7c
31789 Hameln
Tel: 05151 9468 0

Bitstore IT-Consulting GmbH
Zentrale - Berlin Lichtenberg
Frankfurter Allee 285
10317 Berlin
Tel: 030 453 087 80

CBS IT-Service - Bitstore Kaulsdorf UG
Tel: 030 453 087 880 1

Büro Dallgow-Döberitz
Tel: 03322 507 020

Büro Kloster Lehnin
Tel: 033207 566 530

PCE IT-Service - Bitstore Darmstadt UG
Darmstadt
Tel: 06151 392 973 0

Büro Neuruppin
Tel: 033932 606 090

ACI EDV Systemhaus - Bitstore Dresden GmbH
Dresden
Tel: 0351 254 410

Das Systemhaus - Bitstore Magdeburg GmbH
Magdeburg
Tel: 0391 636 651 0

Allerdata.IT - Bitstore Wittenberg GmbH
Wittenberg
Tel: 03491 876 735 7

Büro Liebenwalde
Tel: 033054 810 00

HSA - das Büro - Bitstore Altenburg UG
Altenburg
Tel: 0344 784 390 97

Bitstore IT – Consulting GmbH
NL Piesteritz 
Piesteritz
Tel: 03491 644 868 6

Solltec IT-Services - Bitstore Braunschweig UG
Braunschweig
Tel: 0531 206 068 0

MF Computer Service - Bitstore Gütersloh GmbH
Gütersloh
Tel: 05245 920 809 3

Firmensitz: AM-Soft IT-Service - Bitstore Hameln GmbH i.G. , Brandenburger Str. 7c , 31789 Hameln
Geschäftsführer Janine Galonska