45 lines
6.8 KiB
Plaintext
45 lines
6.8 KiB
Plaintext
2024-04-01 11:35:40,147 root INFO: Starting Hasaki Crawler.......
|
|
2024-04-01 11:35:40,147 root INFO: Loading config file.......
|
|
2024-04-01 11:35:40,147 root INFO: Config file loaded.......
|
|
2024-04-01 11:35:40,147 root INFO: {'crawler_name': 'raena_crawler_engine_hasaki', 'crawler_schema': 'raena_spider_management', 'category_tab': 'rce_category', 'tracker_tab': 'crawler_tracker_hasaki', 'product_tab': 'rce_product', 'variant_tab': 'rce_product_variant', 'brand_tab': 'rce_brand', 'reseller_tab': 'rce_reseller', 'reseller_store_tab': 'rce_reseller_store', 'review_tab': 'rce_ratings_reviews', 'review_productmodels_tab': 'rce_ratings_reviews_productmodels', 'review_producttags_tab': 'rce_ratings_reviews_producttags', 'review_tags': 'rce_tags', 'source_tab': 'rce_source', 'seo_tab': 'rce_seo', 'product_per_category': '1000', 'source_category': '11043145', 'db_user': 'dbadmin', 'db_pass': '5qCif6eyY3Kmg4z', 'database': 'analytics', 'db_host': 'redshift-cluster-1.cdqj58hfx4p7.ap-southeast-1.redshift.amazonaws.com', 'db_port': '5439', 'crawler_main': '1', 'crawler_slave_no': ''}
|
|
2024-04-01 11:35:40,151 root INFO: ++++++++++++++++ process tracker tab status ++++++++++++++++++++++
|
|
2024-04-01 11:35:40,151 root INFO: [('category', 1), ('category_product', 1), ('product_info', 0)]
|
|
2024-04-01 11:35:40,151 root INFO: Category collection completed........ Moving to collecting products ;ist of the categories.......
|
|
2024-04-01 11:35:40,151 root INFO: Category products collection completed........ Moving to collecting product info.......
|
|
2024-04-01 11:35:40,151 root INFO: Initializing HasakiProductInfo
|
|
2024-04-01 11:35:43,844 root INFO: Error:
|
|
2024-04-01 11:35:43,844 root INFO: Error occurred. Please check config file or the internal SQLLITE DB. Exiting......
|
|
2024-04-01 11:35:47,656 root INFO: Closing connection.....
|
|
2024-04-01 11:37:18,979 root INFO: Starting Hasaki Crawler.......
|
|
2024-04-01 11:37:18,979 root INFO: Loading config file.......
|
|
2024-04-01 11:37:18,979 root INFO: Config file loaded.......
|
|
2024-04-01 11:37:18,979 root INFO: {'crawler_name': 'raena_crawler_engine_hasaki', 'crawler_schema': 'raena_spider_management', 'category_tab': 'rce_category', 'tracker_tab': 'crawler_tracker_hasaki', 'product_tab': 'rce_product', 'variant_tab': 'rce_product_variant', 'brand_tab': 'rce_brand', 'reseller_tab': 'rce_reseller', 'reseller_store_tab': 'rce_reseller_store', 'review_tab': 'rce_ratings_reviews', 'review_productmodels_tab': 'rce_ratings_reviews_productmodels', 'review_producttags_tab': 'rce_ratings_reviews_producttags', 'review_tags': 'rce_tags', 'source_tab': 'rce_source', 'seo_tab': 'rce_seo', 'product_per_category': '1000', 'source_category': '11043145', 'db_user': 'dbadmin', 'db_pass': '5qCif6eyY3Kmg4z', 'database': 'analytics', 'db_host': 'redshift-cluster-1.cdqj58hfx4p7.ap-southeast-1.redshift.amazonaws.com', 'db_port': '5439', 'crawler_main': '1', 'crawler_slave_no': ''}
|
|
2024-04-01 11:37:18,983 root INFO: ++++++++++++++++ process tracker tab status ++++++++++++++++++++++
|
|
2024-04-01 11:37:18,983 root INFO: [('category', 1), ('category_product', 1), ('product_info', 0)]
|
|
2024-04-01 11:37:18,983 root INFO: Category collection completed........ Moving to collecting products ;ist of the categories.......
|
|
2024-04-01 11:37:18,983 root INFO: Category products collection completed........ Moving to collecting product info.......
|
|
2024-04-01 11:37:18,983 root INFO: Initializing HasakiProductInfo
|
|
2024-04-01 11:37:21,796 root INFO: Closing connection.....
|
|
2024-04-01 11:37:37,443 root INFO: Starting Hasaki Crawler.......
|
|
2024-04-01 11:37:37,443 root INFO: Loading config file.......
|
|
2024-04-01 11:37:37,444 root INFO: Config file loaded.......
|
|
2024-04-01 11:37:37,444 root INFO: {'crawler_name': 'raena_crawler_engine_hasaki', 'crawler_schema': 'raena_spider_management', 'category_tab': 'rce_category', 'tracker_tab': 'crawler_tracker_hasaki', 'product_tab': 'rce_product', 'variant_tab': 'rce_product_variant', 'brand_tab': 'rce_brand', 'reseller_tab': 'rce_reseller', 'reseller_store_tab': 'rce_reseller_store', 'review_tab': 'rce_ratings_reviews', 'review_productmodels_tab': 'rce_ratings_reviews_productmodels', 'review_producttags_tab': 'rce_ratings_reviews_producttags', 'review_tags': 'rce_tags', 'source_tab': 'rce_source', 'seo_tab': 'rce_seo', 'product_per_category': '1000', 'source_category': '11043145', 'db_user': 'dbadmin', 'db_pass': '5qCif6eyY3Kmg4z', 'database': 'analytics', 'db_host': 'redshift-cluster-1.cdqj58hfx4p7.ap-southeast-1.redshift.amazonaws.com', 'db_port': '5439', 'crawler_main': '1', 'crawler_slave_no': ''}
|
|
2024-04-01 11:37:37,447 root INFO: ++++++++++++++++ process tracker tab status ++++++++++++++++++++++
|
|
2024-04-01 11:37:37,447 root INFO: [('category', 1), ('category_product', 1), ('product_info', 0)]
|
|
2024-04-01 11:37:37,447 root INFO: Category collection completed........ Moving to collecting products ;ist of the categories.......
|
|
2024-04-01 11:37:37,447 root INFO: Category products collection completed........ Moving to collecting product info.......
|
|
2024-04-01 11:37:37,447 root INFO: Initializing HasakiProductInfo
|
|
2024-04-01 11:37:40,69 root ERROR: [Errno 2] No such file or directory: 'Xvfb'
|
|
2024-04-01 11:37:40,72 root INFO: Closing connection.....
|
|
2024-04-01 11:39:24,935 root INFO: Starting Hasaki Crawler.......
|
|
2024-04-01 11:39:24,935 root INFO: Loading config file.......
|
|
2024-04-01 11:39:24,935 root INFO: Config file loaded.......
|
|
2024-04-01 11:39:24,935 root INFO: {'crawler_name': 'raena_crawler_engine_hasaki', 'crawler_schema': 'raena_spider_management', 'category_tab': 'rce_category', 'tracker_tab': 'crawler_tracker_hasaki', 'product_tab': 'rce_product', 'variant_tab': 'rce_product_variant', 'brand_tab': 'rce_brand', 'reseller_tab': 'rce_reseller', 'reseller_store_tab': 'rce_reseller_store', 'review_tab': 'rce_ratings_reviews', 'review_productmodels_tab': 'rce_ratings_reviews_productmodels', 'review_producttags_tab': 'rce_ratings_reviews_producttags', 'review_tags': 'rce_tags', 'source_tab': 'rce_source', 'seo_tab': 'rce_seo', 'product_per_category': '1000', 'source_category': '11043145', 'db_user': 'dbadmin', 'db_pass': '5qCif6eyY3Kmg4z', 'database': 'analytics', 'db_host': 'redshift-cluster-1.cdqj58hfx4p7.ap-southeast-1.redshift.amazonaws.com', 'db_port': '5439', 'crawler_main': '1', 'crawler_slave_no': ''}
|
|
2024-04-01 11:39:24,939 root INFO: ++++++++++++++++ process tracker tab status ++++++++++++++++++++++
|
|
2024-04-01 11:39:24,939 root INFO: [('category', 1), ('category_product', 1), ('product_info', 0)]
|
|
2024-04-01 11:39:24,939 root INFO: Category collection completed........ Moving to collecting products ;ist of the categories.......
|
|
2024-04-01 11:39:24,939 root INFO: Category products collection completed........ Moving to collecting product info.......
|
|
2024-04-01 11:39:24,939 root INFO: Initializing HasakiProductInfo
|
|
2024-04-01 11:39:27,536 root ERROR: [Errno 2] No such file or directory: 'Xvfb'
|
|
2024-04-01 11:39:27,538 root INFO: Closing connection.....
|