[Friedx]

There is always a challenge when dealing with large CSV and Excel files, as even MS Office and other software cannot open files with more than 1 million rows. In such situations, this Python script offers a solution by allowing users to extract specific columns from large CSV or Excel files, preview their content, and export them to plain text files with ease. It’s perfect for quickly parsing and organizing datasets.


Features: 
1. Handles Large Files: 
   - Processes large CSV files using chunked loading to avoid memory issues. 

2. Interactive Column Selection: 
   - Lists all available columns from the dataset. 
   - Previews the first five rows of the selected column before exporting. 

3. Multi-File Format Support: 
   - Compatible with .csv .xls, and .xlsx file formats. 

4. Error Handling: 
   - Automatically skips problematic lines in CSV files. 
   - Issues warnings for empty columns. 

5. Custom Export Options: 
   - Saves selected columns to text files with a user-defined filename. 



Libraries Used: 
- os: Provides operating system interaction (not actively used here). 
- pandas: For reading and processing CSV/Excel files. 
- tkinter: For file selection and save dialogs through a GUI interface. 



Installation Instructions: 
To install the required libraries, use this single command: 

pip install pandas tkinter

How to Use the Tool: 
1. Save the script as a .py file, for example, column_extractor.py  
2. Open a terminal or command prompt and navigate to the directory where the script is saved. 
3. Run the script using the command:  python column_extractor.py

import os import pandas as pd from tkinter import filedialog, Tk # ASCII Art for Tool tool_name = r""" ////////////////////////////////////////////////////////////////////////////////////////////// //      ____      _                _____      _                  _    ____                // //    / ___|___ | |_  _ _ __ ___ | ____|_  _| |_ _ __ __ _  ___| |_  |  _ \ _ __ ___      // //    | |  / _ \| | | | | '_ ` _ \|  _| \ \/ / __| '__/ _` |/ __| __| | |_) | '__/ _ \    // //    | |__| (_) | | |_| | | | | | | |___ >  <| |_| | | (_| | (__| |_  |  __/| | | (_) |    // //    \____\___/|_|\__,_|_| |_| |_|_____/_/\_\\__|_|  \__,_|\___|\__| |_|  |_|  \___/    // ////////////////////////////////////////////////////////////////////////////////////////////// """ # Display tool name and tagline print(tool_name) print("By FRIEDX ") # Function to select file(s) def select_file():     root = Tk()     root.withdraw()  # Hide the Tkinter window     file_path = filedialog.askopenfilename(         title="Select a CSV or Excel file",         filetypes=[("CSV Files", "*.csv"), ("Excel Files", "*.xls;*.xlsx")],     )     return file_path # Function to read the file def read_file(file_path):     try:         if file_path.endswith('.csv'):             # Read CSV file in chunks             chunks = pd.read_csv(file_path, chunksize=100000, on_bad_lines='warn', engine='python')             return pd.concat(chunks, ignore_index=True)         elif file_path.endswith(('.xls', '.xlsx')):             return pd.read_excel(file_path)         else:             print("Unsupported file type.")             return None     except Exception as e:         print(f"An error occurred while reading the file: {e}")         return None # Function to list and select columns def list_columns(df):     columns = df.columns.tolist()     print("\nAvailable Columns:")     for i, col in enumerate(columns, 1):         print(f"{i}. {col}")     return columns # Function to preview the selected column def preview_column(df, column_name):     preview = df[column_name].head(5)     print(f"\nPreview of column '{column_name}':")     print(preview)     return preview # Function to export the column data to a text file def export_column(df, column_name):     # Check if column is empty     if df[column_name].isnull().all():         print(f"Warning: The column '{column_name}' is empty.")         return         # Prompt user for save location     file_name = f"{column_name}.txt"     save_path = filedialog.asksaveasfilename(         defaultextension=".txt",         initialfile=file_name,         title="Save File As",         filetypes=[("Text Files", "*.txt")]     )     if not save_path:         print("Export cancelled.")         return     df[column_name].to_csv(save_path, index=False, header=False, sep="\n")     print(f"Column '{column_name}' has been exported to {save_path}.") # Function to handle column selection, preview, and export def handle_column_export(df, columns):     while True:         try:             column_choice = int(input("\nEnter the serial number of the column to export: "))             if column_choice < 1 or column_choice > len(columns):                 print("Invalid choice. Please enter a valid serial number.")                 continue                         column_name = columns[column_choice - 1]             preview_column(df, column_name)                         proceed = input(f"\nDo you want to export the column '{column_name}'? (y/n): ").lower()             if proceed == 'y':                 export_column(df, column_name)             elif proceed != 'n':                 print("Invalid input. Please enter 'y' or 'n'.")                 continue                         # Ask if the user wants to extract another column or exit             while True:                 next_action = input("\nDo you want to extract another column? (y/n): ").lower()                 if next_action in ['y', 'n']:                     break                 print("Invalid input. Please enter 'y' or 'n'.")                         if next_action == 'n':                 print("Exiting the tool. Goodbye!")                 break         except ValueError:             print("Invalid input. Please enter a valid number.")         except Exception as e:             print(f"An error occurred: {e}") # Main function to start the tool def main():     file_path = select_file()     if not file_path:         print("No file selected. Exiting.")         return     print(f"\nProcessing file: {file_path}")         # Read the file     df = read_file(file_path)     if df is None:         return         # List columns     columns = list_columns(df)     # Handle column selection and export     handle_column_export(df, columns) # Run the tool if __name__ == "__main__":     main()

Credits:  
Developed by FRIEDX.  


Let me know if you need further edits!

[darkwurm]

Very nice. You could actually handle xls/xlsx on a system with gnumeric installed. If you run ssconvert (which is installed as part of gnumeric) you can convert them to csvs. in python you could call it it with shell execution. ie subprocess.Popen('ssconvert xlsfilename csvfilename, shell=True) then run it through your main logic.

[salv]

cool very nice thanks bro

[Friedx]

If any suggestion to improve this script let me know. thank you