[Apps] Tools Import Data

[ClicShopping]

Hello,

I take your update and make little modification, I tested on osCommerce and phoenix, works fine. I try with oscMax, but there is a problem on the db importation.

Let me know if you have a problem.

Thank you for your contribution.

 

https://github.com/ClicShoppingOfficialModulesV3/apps_tools_import_data

[AlwaysSkint]

Can you be more specific with your osCmax issue? (It's not OscMax).

 

You may wish to add a note to the readme.md

"Larger stores may need to temporarily extend PHP memory and timeout - 256MB and 120 (seconds) should cater for most stores. This is often done by adjust php.ini or .user.ini"

 

Thanks.

Edited November 8, 2020 by AlwaysSkint

[AlwaysSkint]

Typo above. "adjust" --> "adjusting"

[ClicShopping]

I do not say there is issue, upload the latest and just retest please.

[AlwaysSkint]

I notice that you haven't adopted my changes to ImportData

• Prevent duplicate key errors when resubmitting a migration and ensure no old orders (for example) remain. The current truncates seem arbitrary.
• Prevent the default Logo banner from being erased (it's not obvious how to recreate it).

[AlwaysSkint]

55 minutes ago, ClicShopping said:

I try with oscMax, but there is a problem on the db importation.

This is why I asked.

Note: I forgot to test 'newsletters' I'll check that for version 1.1

Edited November 8, 2020 by AlwaysSkint

[ClicShopping]

Hello,

 

Do you have a file to send me,I can test ?

I updated and Put code to increase the memory limit, remove the time out.

Remove the banner migration

Add contributor file

 

Thank you

[AlwaysSkint]

I'll have a look around my local repos for v2.0x demo data - I can't send you my client's data.

Once I have v2.0x (nearly) finalised, I'll begin osCmax 2.5 testing - there were a few db changes though nothing major.

[AlwaysSkint]

Do note that the following may be restricted in some server environments:

      ini_set('memory_limit','256M');
      set_time_limit(0);

This is likely to throw a 500 error.

[AlwaysSkint]

Import to a newly created db..

osCmax 2.0.25 Demo Data

[ClicShopping]

Yes you have right. I will remove that and write little information on that inside the readme

Thank you for the file, I uploaded and make a test more later. Tk

Also, You remove your file on github, normal ?

[AlwaysSkint]

Yes, I did say that I couldn't understand github. 😢

I'm trying to just have one repo called AlwaysSkint, that pulls updates from your master when it gets updated. Then I can make proposed alterations to my fork/branch whatever.

I can't even figure out how to delete my fork and start again! I'm getting too old/stupid for this. 😢

Edited November 8, 2020 by AlwaysSkint

[AlwaysSkint]

Added a comment on one of the commits, regarding banners - I think that you may have misunderstood me.

(Managed to delete & remake my github repo.)

Edited November 8, 2020 by AlwaysSkint

[AlwaysSkint]

It would be extremely useful if the default logo ID could be changed to say 1000.

This should have no impact on any new stores and will allow migrations to keep existing banners. Even for existing stores, there should be no (or very rare) issues.

UPDATE `banners` SET `banners_id` = '1000' WHERE `banners`.`banners_id` = 1;

 

Edited November 8, 2020 by AlwaysSkint

[Patrick]

Hello,

 

I do not like this idea because you do not know if a store owner will create lot of banner or not.

It's better to remove everything in my sense.

[AlwaysSkint]

The banner ID is assigned INT(11), so there is very little danger of a store owner using up the number of possible banners! The most banners that I have seen used in a large store (10s of thousands of products) has been about 25 and it begins to look silly. 😉

 

About to attempt to submit v1.1 osCmax20.php plus revert some of your changes.. If I can get it to work.

 

 

Edited November 8, 2020 by AlwaysSkint

[AlwaysSkint]

I'm looking for a little bit of help here.. Customer groups wont migrate due to a slightly different table definition, so I'm trying to temporarily alter it but it returns a mysql syntax error.

//******************************************
//Customers Groups
//******************************************
/* Inverted restrictions

          'group_payment_allowed' => $data['group_payment_allowed'],
          'group_shipment_allowed' => $data['group_shipment_allowed']
*/
// Set default to match original db
      $Qalter = $this->db->prepare('ALTER TABLE :customers_groups 
      								CHANGE customers_group_id customers_group_id INT(11) NOT NULL DEFAULT 0
                                   ');
      $Qalter->execute();
      echo 'temporary structure change<br>';
      
      $QcustomersGroups = $mysqli->query('select *
                                         from ' . $this->PrefixTable . 'customers_groups
                                       ');
      echo '<hr>';
      echo '<div>table_customers_groups</div>';
      echo '<div>' . CLICSHOPPING::getDef('text_number_of_item') . ' : ' . $QcustomersGroups->num_rows . '</div>';
      echo '<hr>';

      while ($data = $QcustomersGroups->fetch_assoc()) {
        $sql_data_array = [
          'customers_group_id' => (int)$data['customers_group_id'],
          'customers_group_name' => $data['customers_group_name'],
          'group_tax' => $data['customers_group_tax_exempt'],
          'group_order_taxe' => (int)$data['customers_group_show_tax']
        ];

        $this->db->save('customers_groups', $sql_data_array);
      }

// Return structure to original
      $QalterRevert = $this->db->prepare('ALTER TABLE :customers_groups 
      									  CHANGE customers_group_id customers_group_id INT(11) NOT NULL AUTO_INCREMENT
                                  		 ');

      $QalterRevert->execute();

 

[Patrick]

You can look clicshopping_v2 maybit can help you

https://github.com/ClicShoppingOfficialModulesV3/apps_tools_import_data/blob/7f8c544a8e1742d26a1f549ca946665ad0200d3e/includes/Module/Hooks/ClicShoppingAdmin/ImportDatabase/ClicShoppingV2.php

line around 548

 

In my sense it's better to make an update via Customer/Customer group  and add other groups after.

 

customers_group_id int not_null auto_increment
customers_group_name varchar(32) not_null
customers_group_discount decimal(11,2) default(0.00) not_null
color_bar varchar(8) default('#FFFFFF') not_null
group_order_taxe tinyint(1) default(0) not_null
group_payment_unallowed varchar(255) default('cc')
group_shipping_unallowed varchar(255)
group_tax varchar(5) default('false') not_null
customers_group_quantity_default int(4) default(0) not_null
--
primary customers_group_id
idx_customers_group_name customers_group_name
##
engine innodb
character_set utf8mb4
collate utf8mb4_unicode_ci

 

Maybe your problem is the primary.

Edited November 8, 2020 by Patrick

[AlwaysSkint]

It is the primary - has autoincrement and a different default. This appears to cause a corruption during migration.

[Foster]

Hello,

 

you can try this

 

      if ($data['customers_group_id'] > 0) {  
        $sql_data_array = [
          'customers_group_id' => (int)$data['customers_group_id'],
          'customers_group_name' => $data['customers_group_name'],
          'group_tax' => $data['customers_group_tax_exempt'],
          'group_order_taxe' => (int)$data['customers_group_show_tax']
        ];
      }

 

[AlwaysSkint]

@Foster Thanks for trying to help but that is essentially what I was doing when I discovered the data corruption. Hence looking to change the table structure. I'm not looking to append to the existing data, which is irrelevant - it's required to replace it.

Fundamentally, the source database begins customers_group_id at zero, whereas ClicShopping begins at 1.

I'm now at the stage of dropping the table and trying to recreate it. After a few hours, I've discovered "OM" appears to have a dbdelete function but not a simple dbcreate function. This got me looking into creating a custom\schema but seems overly complex for something quite simple. 😕

I'll perhaps backtrack and reanalyse the problem.

 

 

Edited November 8, 2020 by AlwaysSkint

[AlwaysSkint]

If I manually remove AUTO_INCREMENT from the target customers_groups table, then the migration works. So I need a simple means of removing this prior to the migration, then add it back in once the data has transferred.

 

Edited November 8, 2020 by AlwaysSkint

[AlwaysSkint]

I've now figured out this was the wrong way around and now corrected to..

          'group_tax' => (int)$data['customers_group_show_tax'],
          'group_order_taxe' =>  $data['customers_group_tax_exempt']

There is an issue with ClicShopping

'group_tax' = 0 should be Excluding Tax and 'group_tax' = 1 should be Including Tax.

Also, I can't use 2 other fields because they are the opposite of the original meaning..

Originals are: 'group_payment_allowed' and 'group_shipment_allowed'

ClicShopping: 'group_payment_unallowed' and 'group_shipment_unallowed'

😢

 

Edited November 8, 2020 by AlwaysSkint

[AlwaysSkint]

@Patrick A quick mention that I have an open Pull request with some updates, plus some English corrections on ClicShopping itself.

Edited November 9, 2020 by AlwaysSkint

[ClicShopping]

Hello,

 

Your english update has been integrated in the next release

Tk