Quick-n-Dirty python3 tools => gtools.py

gtools.py

A Quick-n-Dirty Python3 Toolbox

gtools.py can be downloaded from: (https://github.com/geoffmcnamara/gmodules)

I wrote these tools over the past many years. Many others deserve credit as I have learned much from so many.

My desire is to have a set of tools available to me from one file that can get frequently needed functions done for me without hassle or having to look things up all the time. Over the years these tools have evolved into fairly decent and flexible code blocks that also satisfy my need for a bit of color and ANSI-coded boxes.

The “rich” module textualize/rich came close but in some areas I need a different method. Please understand that rich is a far more serious set of tools that offers many more features and has been fully tested. I like to center things on the screen and my tools make that easy. Color coding is easy and I borrowed the idea of using color code tagging from the rich tool set.

There are over 100 functions in this module.

You can get a sense of its power by running it… but it’s real power is importing it and using the tools. There are a lot of tricks in these tools which can expose a novice programmer to quick ways to debug and tweak their own code.

To see some tools:

./gtools.py

To use gtools in your own code:

from gtools import dbug, docvars, printit, gselect, gtable, cat_file, remap_keys, quick_plot  # etc, etc...

I welcome any feedback, good, bad, or ugly. The code is still developing and could obviously use improvement. It has glaring errors and poor code styles so your input would be to everyone’s benefit.

Life is good, relationships are everything, working together insures the future.

Enjoy,

-geoff- geoff.mcnamara@gmail.com

For reference, here is a list of functions and docs:

--------------
add_content:          I wrote this because I am constantly building csv files with a header line                           
                      consider add_or_replace() function                                                                   
                      Required:                                                                                            
                      file                                                                                                 
                      content=str|list                                                                                     
                      Options:                                                                                             
                      after=pattern                                                                                        
                      before=pattern                                                                                       
                      replace=pattern                                                                                      
                      position=##                                                                                          
                      if none of those content is appended to the file                                                     
                      if header is also included it will be added to the begining of the file if it does not already exitst
                      used_to_be: add_line()                                                                               
--------------
add_or_replace:       purpose: Adds or replaces a line in a file where pattern occurs                             
                      required: filename, action: str [before|after|replace|either] ,pattern, new_line            
                      action: before|after|replace|either (either will replace if it is found or add if it is not)
                      options:                                                                                    
                      -   backup: bool=True,                                                                      
                      -   ask: bool=False,                                                                        
                      -   centered: bool=False,                                                                   
                      -   shadowed: bool=False                                                                    
                      pattern: needs to be unique regex?                                                          
                      returns: "done" or None depending on use                                                    
--------------
allmax:               purpose: justifies using nclen all strs in msgs_l and maximizes each string length to the longest string
                      input: msg_l: list, justify: str ('left'|'center'|'right')                                              
                      output: new_msgs: list of strngs justified                                                              
--------------
askYN:                auto var can be used to automatically invoke the default                                               
                      TODO: timeout=10  # times out in 10 secs and invokes dflt - this still requires you to hit enter : TODO
                      # >>> askYN()                                                                                          
                      # Continue [y]: True                                                                                   
--------------
bool_val:             s can be a str or list                                                                                   
                      args_l must be provided                                                                                  
                      kvargs is optional                                                                                       
                      used to see if a string or a list of stings might be declared true                                       
                      by being in args or seeing it has a bool value set in kvargs                                             
                      return: bool_val (False unless stipulated to be otherwise or declared in kwargs with 'dflt' or 'default')
                      use:                                                                                                     
                      DBUG = bool_val('dbug', args, kvargs)                                                                    
                      or                                                                                                       
                      DBUG = bool_val(['dbug', 'DBUG'], args, kvargs)                                                          
--------------
boxed:                purpose: draw a unicode box around msgs                            
                      args: msgs                                                         
                      options:                                                           
                      centered | center: bool  # centers box on the screen               
                      txt_center: int  # num of lines from top to center in the box      
                      color: str  # text color                                           
                      box_color: str  # color of border                                  
                      title, footer: str # goes in topline or bottom line centered of box
                      width forces the width size defaults to # screen columns           
                      shadowed | shadow: bool # adds a shadow right and bottom           
                      ... some other options; see below                                  
                      returns boxed lines: list                                          
                      NOTES: this function does not print - it returns the box lines     
--------------
browseit:             purpose: opens the url in your browser
                      requires: url: str                    
                      returns: none                         
--------------
cat_file:             purpose: reads a file and return lines or rows_lol (list of list) or as a df (dataframe)                       
                      options:                                                                                                       
                      -    prnt: bool,  # prints out the file contents                                                               
                      -    lst: bool,   # returns a list of lines or you could use: txt.split('                                      
                      ') to make it a list                                                                                           
                      -    csv: bool,   # treat the file as a csv (or for me, a dat file)                                            
                      -    xlsx: bool,  # returns df of a spreadsheet file                                                           
                      -    hdr: bool,   # whether to include header line or header data in the return                                
                      -    df: bool,    # return a df                                                                                
                      -    rtrn: str, (can be "str", "string", "lst", "df"                                                           
                      returns the text of a file as a str or rows_lol (if it is a cvs: bool file) or returns a df if requested       
                      Note: if the result df has the header/colnames repeated in row[0] then make sure you included 'hdr' or hdr=True
                      #>>> t = cat_file("/etc/timezone")                                                                             
                      #>>> print(t)                                                                                                  
                      America/New_York                                                                                               
                      <BLANKLINE>                                                                                                    
--------------
centered:             purpose: calculates screen placement for msgs: list|str
                      options:                                               
                      length=columns: int                                    
                      shift=0: int                                           
                      'str'|'string'=False: bool                             
                      'lst'|'list'=True: bool                                
                      returns: line|lines                                    
                      note: replaces deprecated centerit()                   
--------------
chk_substr:           purpose: given a list of strings to check (chk_l) and a list of substrings to compare
                      if any "compare" substring is in any string in check list of strings then            
                      do action either 'exclude' or 'include'                                              
                      return: new_list                                                                     
--------------
cinput:               aka: centered input                                                                          
                      purpose: gets input from user using provided promt - centers the prompt on the screen        
                      options:                                                                                     
                      - shift: int   # allows you to shift the position of the prompt (from the center) eg shift=-5
                      - quit|exit|close: bool   # will quit with do_close() if availaable or just sys.exit()       
                      returns: user response                                                                       
--------------
clr_coded:            purpose: takes any msg string containing tags for colors and decodes them into a colorized string
                      requires: msg_s: str                                                                             
                      returnd: colorized string                                                                        
                      decode a string - replace \[color\].*[\/] with code and reset                                    
                      requires a space before the first bracket                                                        
--------------
cls:                  Clears the terminal screen.
--------------
color_neg:            purpose: this conditions (colorizes and adds commas) elems if they are numbers                    
                      input: elem                                                                                       
                      options: "neg_color=red on black!": str, pos_color="green! on black!": str, rnd=0: int            
                      -   color: bool        # will color the number ... default red for negative and green for positive
                      -   neg_color: str     # you can change the color for negative numbers                            
                      -   pos_color: str     # you can change the color for positive numbers                            
                      -   human: bool        # adds commas, reduces large numbers to 10000000 to 1M etc                 
                      -   nan: str           # allows you to change "nan" or "NaN" to any string you want. default=""   
                      returns: elem (conditioned; colored)                                                              
                      use:                                                                                              
                      -   for n, row in enumerate(lol):                                                                 
                      -       ...                                                                                       
                      -       if neg:                                                                                   
                      -           row = [color_neg(elem) for elem in row]                                               
                      -       # table.add_row(*row)                                                                     
                      -       table_lol.append(*row)                                                                    
                      NOTE: this may return an elem with a different length                                             
--------------
color_neg:            purpose: this conditions (colorizes and adds commas) elems if they are numbers                    
                      input: elem                                                                                       
                      options: "neg_color=red on black!": str, pos_color="green! on black!": str, rnd=0: int            
                      -   color: bool        # will color the number ... default red for negative and green for positive
                      -   neg_color: str     # you can change the color for negative numbers                            
                      -   pos_color: str     # you can change the color for positive numbers                            
                      -   human: bool        # adds commas, reduces large numbers to 10000000 to 1M etc                 
                      -   nan: str           # allows you to change "nan" or "NaN" to any string you want. default=""   
                      returns: elem (conditioned; colored)                                                              
                      use:                                                                                              
                      -   for n, row in enumerate(lol):                                                                 
                      -       ...                                                                                       
                      -       if neg:                                                                                   
                      -           row = [color_neg(elem) for elem in row]                                               
                      -       # table.add_row(*row)                                                                     
                      -       table_lol.append(*row)                                                                    
                      NOTE: this may return an elem with a different length                                             
--------------
convert_temp:         expects a string with either an ending.lower() of "f" or "c" to declare what to return
                      returns rounded(converted_temp)                                                       
                      always returns a string with 2 places (inlcuding 0s)                                  
--------------
dbug:                 To test run: python3 -m doctest -v dbug.py
                      # >>> a = "xyz"                           
                      # >>> dbug_var(a)                         
                      DEBUG: [dbug.py; <module>:1] a:xyz        
                      '1'                                       
--------------
ddbug:                purpose: this is for use by dbug only... as dbug can't call itself
--------------
display_in_cols:      deprecated as gtable may take it's place                     
                      input: takes a list or a dict and prints in order across cols
                      options:                                                     
                      order default for list is is as-is                           
                      if it is a dict                                              
                      you can sort "on" default "value" or on "key"                
                      , or reverse=True or reverse=False                           
                      returns: lines (sorted if dict)                              
                      example:                                                     
                      # >>> lst = [*range(1,20)]                                   
                      # >>> display_in_cols(lst)                                   
                      then print lines eg printit(lines)                           
--------------
do_close:             purpose: to provide a boxed closing message                           
                      returns: None                                                         
                      # dflt_msg = "Enjoy!"                                                 
                      input msg or it uses dflt_msg                                         
                      options: 'center' | 'centered' color='red' rc=return code to exit with
                      >>> do_close()                                                        
                      ======                                                                
                      Enjoy!                                                                
                      ======                                                                
--------------
do_edit:              purpose: launches vim editor with file           
                      a quick-n-dirty utility to edit a file           
                      options:                                         
                      - lnum: line number                              
                      Initiate edit on a file - with lineno if provided
--------------
do_func_docs:         purpose: to get funtion docs
--------------
do_kv_rows:           deprecated, use kv_cols instead                                                                                   
                      purpose:                                                                                                          
                      prints out boxed dictionary of key value pairs                                                                    
                      # >>> d = {"one": 1.00, "two": "2.000", "three": 3.00, "four is where the world changes for the better": "4.4444"}
                      # >>> lines = do_kv_rows(d, length=21)                                                                            
--------------
do_list:              # >>> lst = ['one','two','three']
                      # >>> do_list(lst, "nc")         
                      # ===========                    
                      # || one   ||                    
                      # || two   ||                    
                      # || three ||                    
                      # ===========                    
--------------
do_logo:              require: nothing but you should provide some default content: str|list                                                                 
                      option: content: str|list, prnt: bool, figlet: bool, center: bool, shadow: bool, box_color: str, color: str, fortune: bool, quote: bool
                      fotune: bool <-- requires the fortune app                                                                                              
                      quote: str   <-- requires a filename with quote lines within it - one will be randomly selected                                        
                      if content = "" and /usr/local/etc/logo.nts does not exist then we use "Your Organization Name"                                        
                      if content == "" then open default file: /usr/local/etc/logo.nts                                                                       
                      if content is a filename then use the lines in that file                                                                               
                      if content is a str and not a file then use pyfiglet to turn it into ascii letters of print lines                                      
--------------
do_prcnt_bar:         purpose: displays a percentge bar                                                                                          
                      args: amnt (prcnt)                                                                                                         
                      options: full_range=100 if you submit this prcnt will be based on it                                                       
                      - bar_width=40: int   # declares width of bar                                                                              
                      - color: str          # text color                                                                                         
                      - done_color:         # color of done portion                                                                              
                      - undone_color:       # color of undone portion                                                                            
                      - done_chr: str       # done character                                                                                     
                      - undone_chr: str     # undone character                                                                                   
                      - prompt: str         # prompt (before the bar)                                                                            
                      - suffix: str         # suffix (afterr the bar)                                                                            
                      - brackets=["[", "]"]: list                                                                                                
                      - show_prcnt=True: bool  # include the percentage at the end                                                               
                      - prnt=False: bool  # False to allow use in dashboards  # you can tell this to not print so you can include in in a box etc
                      returns: percent bar line as a str                                                                                         
                      #>>> rh = 56                                                                                                               
                      #>>> rl = 50                                                                                                               
                      #>>> cp = 51                                                                                                               
                      #>>> amnt = cp - rl                                                                                                        
                      #>>> full_range = rh - rl                                                                                                  
                      #>>> print(f"rl {do_prcnt_bar(amnt,full_range)} rh")                                                                       
                      # rl [██████----------------------------------]16% rh                                                                      
--------------
do_title:             deprecated and will be removed                         
                      20210208-1203 significant refactoring done             
                      A quick-n-dirty utility to print out a title line using
                      title = as the title string                            
                      c = as the repeated character                          
                      length defaults to 120                                 
                      color defaults to ""                                   
                      do_title(title, hchr, length, color)                   
                      # >>> do_title("mytitle","-",40,color="")              
                      --------------- mytitle ----------------               
                      '--------------- mytitle ----------------'             
--------------
do_title_three_line:  This is deprecated and will be removed                              
                      Use: do_title_three_line(msg,length=120)                            
                      This should probably be made into a class and combined with do_title
                      # >>> do_title_three_line("mytitle",30)                             
                      ==============================                                      
                      mytitle                                                             
                      ==============================                                      
                      ['==============================',                                  
                      '           mytitle',                                               
                      '==============================']                                   
--------------
docvars:              purpose: wrapper for function to allow variable substitution within its doc
                      I use this in front on a function I call handleOPTS(args)                  
                      which works with the module docopts                                        
                      Thank you to stackoverflow.com: answered Apr 25 '12 at 1:54 senderle       
                      this is a very useful way to allow variables in your __doc__ strings       
                      wrap a function with this and use if this way                              
                      eg:                                                                        
                      @docvars(os.path.basename(__file__), anotherarg, myarg="abcd")             
                      def myfunc():                                                              
                      """                                                                        
                      Usage: {0} [-hijk]"                                                        
                      Notes:                                                                     
                      anotherarg: {1}                                                            
                      myarg: {myarg}                                                             
                      """                                                                        
                      return "Done"                                                              
--------------
escape_ansi:          purpose: Removes ansii codes from a string (line)                    
                      aka: name should be escape_ansii                                     
                      returns: "cleaned no_code" line                                      
                      TODO: allow option to return clr_codes[1], nocode(elem), clr_codes[2]
                      see: split_codes()                                                   
--------------
file_exists:          Note: type can be "file" or "dir" (if it isn't file the assumption is dir)
                      should rename this to path_exists... argghh                               
                      #>>> file_exists('/etc/hosts')                                            
                      True                                                                      
--------------
find_file_in_dirs:    purpose: this is for future use as something like find_file_in_dirs(filename, dirs_l)
                      if dirs_l is empty it defaults to ["./"]                                             
--------------
first_matched_line:   purpose: return: just the first matched line(s) from a filename using pattern
                      required:                                                                    
                      - filename: str                                                              
                      - pattern: str                                                               
                      options:                                                                     
                      - upto: int default=1  # How many matching lines to return                   
                      returns: matching lines                                                      
--------------
from_to:              purpose: returns lines from a *file* from BEGIN pattern to END pattern" (between BEGIN and END)       
                      options:                                                                                              
                      -  include: can be equal to "none", top|begin|start, bottom|end, or 'both'                            
                      --    include='top' will include the begin pattern line,                                              
                      --    include='bottom' will include the end pattern line                                              
                      --    include='both' will include top and end pattern matching lines                                  
                      returns: lines between (or including) begin pattern and end pattern from filename (or a list of lines)
--------------
fstring:              purpose: alternative to f-strings
                      use: fsting(s)                   
                      returns: formated s string       
--------------
funcname:             Use: eg:                                    
                      from dbug import dbug, fname                
                      def myfunc():                               
                      dbug(f"This is function name: {funcname()}")
                      pass                                        
--------------
gclr:                 Purpose: to return color code + text (if any) NOTE: sub_color() uses this!                                               
                      input:                                                                                                                   
                      text: str = ""  # if "" then the color code alone is returned                                                            
                      color: str = 'normal'  # examples: 'red on black', 'bold green on normal', 'bold yellow on red', 'blink red on black' etc
                      Notes:                                                                                                                   
                      color is the first argument because you may just want to return the color code only                                      
                      run gcolors.demo() to see all color combinations                                                                         
                      returns: color coded [and text]                                                                                          
--------------
gcolumnize:           purpose: This will columnize (vertically) a list                                   
                      input: msg_l: list|lol, width=0: int, color="": str                                
                      options:                                                                           
                      - sep: str                                                                         
                      - prnt|print|show = False: bool                                                    
                      - centered = False: bool  # only invoked if prnt == True                           
                      - title = "": str   # only invoked if prnt == True                                 
                      - footer = "": str  # only invoked if prnt == True                                 
                      return: lines: list                                                                
                      required: import columnize                                                         
                      If it is a list of lists ( like several boxes made up of lines ) then              
                      it will list them next to each other                                               
                      box1 = +------+                                                                    
                      | box1 |                                                                           
                      +------+                                                                           
                      box2 = +------+                                                                    
                      | box2 |                                                                           
                      +------+                                                                           
                      boxes = box1 + box2                                                                
                      lines = gcolumnize(boxes)                                                          
                      printit(lines)                                                                     
                      +------+  +------+                                                                 
                      | box1 |  | box2 |                                                                 
                      +------+  +------+                                                                 
                      # ### or ### #                                                                     
                      mylist = ["One potato", "Two potato", "Three potato", "Four", "Now close the door"]
                      lines = gcolumnize(mylist, width=40)                                               
                      printit(lines)                                                                     
                      One potato    Four                                                                 
                      Two potato    Now close the door                                                   
                      Three potato                                                                       
--------------
gcolumnize:           purpose: This will columnize (vertically) a list                                   
                      input: msg_l: list|lol, width=0: int, color="": str                                
                      options:                                                                           
                      - sep: str                                                                         
                      - prnt|print|show = False: bool                                                    
                      - centered = False: bool  # only invoked if prnt == True                           
                      - title = "": str   # only invoked if prnt == True                                 
                      - footer = "": str  # only invoked if prnt == True                                 
                      return: lines: list                                                                
                      required: import columnize                                                         
                      If it is a list of lists ( like several boxes made up of lines ) then              
                      it will list them next to each other                                               
                      box1 = +------+                                                                    
                      | box1 |                                                                           
                      +------+                                                                           
                      box2 = +------+                                                                    
                      | box2 |                                                                           
                      +------+                                                                           
                      boxes = box1 + box2                                                                
                      lines = gcolumnize(boxes)                                                          
                      printit(lines)                                                                     
                      +------+  +------+                                                                 
                      | box1 |  | box2 |                                                                 
                      +------+  +------+                                                                 
                      # ### or ### #                                                                     
                      mylist = ["One potato", "Two potato", "Three potato", "Four", "Now close the door"]
                      lines = gcolumnize(mylist, width=40)                                               
                      printit(lines)                                                                     
                      One potato    Four                                                                 
                      Two potato    Now close the door                                                   
                      Three potato                                                                       
--------------
gcontains:            purpose: to determine if any pattern (str|list) is a substring of string (string_s)
                      options: none                                                                      
                      returns: bool                                                                      
--------------
get_boxchrs:          purpose: given a box_style (ansi, single, solid, double) will return a set of chars for creating a box
                      input: box_style: str                                                                                 
                      return:                                                                                               
                      [tl, hc, ts, tr, vc, ls, rs, ms, bl, bs, br]  as a list in the order shown                            
                      Note: boxed() uses this                                                                               
                      tl = top_left, hc=horizontal_char, ts=top_separator, tr=top_right, vc=vertical_char,                  
                      ls=left_separator, rs=right_separator, ms=middle_separator                                            
                      bl=bottom_left, bs=bottom_sep, br=bottom_right                                                        
--------------
get_columns:          gets screen/terminal cols OR rows                                      
                      returns int(columns)| int(rows: bool) | int(cols), int(rows) both: bool
--------------
get_elems:            Input:                                                               
                      lines (as a list) and                                                
                      optionally a delimiter (default is a comma)                          
                      optionally a  col_limit (size)                                       
                      Returns:                                                             
                      an array: list of list (lol - lines of elements aka rows and columns)
--------------
get_html_tables:      input: url, display (tables with tabulate), access defaults to selenium which is slow but gets by bot blocks to url request
                      returns: list of panda dataframes                                                                                          
                      requires:                                                                                                                  
                      import pandas as pd                                                                                                        
                      from selenium import webdriver                                                                                             
                      # from fake_useragent import UserAgent                                                                                     
--------------
get_mod_docs:         purpose: lists all functions and the docs from a module
                      Note: except some functions eg _demo                   
                      returns cnt                                            
--------------
get_random_line:      purpose: grabs one random line from a file                 
                      requires:                                                  
                      from gtools purify_file, centered, boxed, printit, cat_file
                      import random                                              
                      returns: line                                              
                      Note: file has all comments removed first (purified)       
--------------
gline:                args: width: int, msg: str <-- msg has to be a key=val pair! eg: gline(60, msg="My Message", just='center')
                      options: width, lc, rc, fc, box_color, color, pad, lpad, rpad, lfill_color,  rfill_color, just: str        
                      returns: line: str                                                                                         
--------------
grep_lines:           purpose: searches lines for pattern                                                                                 
                      options:                                                                                                            
                      -    ic: bool (insensitive case)                                                                                    
                      -        rtrn_bool=False: bool    #  (whether to rtrn lines [default] or bool result)                               
                      -        csv: bool                # will convert a list of lists to a list of csv style lines before searching      
                      -                                 #   and but returns the line as a list, just the way we got it                    
                      returns: matched line(s) (or True False if rtrn_bool is True)                                                       
                      Note: if only one line is matched then it will return that one line otherwise it will return a list of matched_lines
--------------
gselect:              purpose: menu type box for selecting by index, key, or value                                                                       
                      required:                                                                                                                          
                      - selections: list or dictionary                                                                                                   
                      options:                                                                                                                           
                      - prompt, center, title, footer, dflt, width, box_color, color, show,                                                              
                      - rtrn='k|key' or 'v|val|value' <-- tells gselect whether you want it to return a key or a value from a supplied list or dictionary
                      This is an important option and allows control over what gets returned. See the Notes below.                                       
                      - show="k|key" or 'v|val|value' <-- tells gselect whether to display a list of keys or a list of values                            
                      - quit: bool <-- add "q)uit" to the prompt and will do a sys.exit() if ans in ("q","Q","exit")                                     
                      - multi <-- allows multiple selections and returns them as a list                                                                  
                      - default|dflt='':   # allows you to /declare a default if enter is hit                                                            
                      - quit                                                                                                                             
                      - rtrn="v" str       # return the selected 'k'ey or 'v'alue                                                                        
                      - show='k' str       # wh/ether to show or display 'k'eys or 'v'alues                                                              
                      - title                                                                                                                            
                      - footer                                                                                                                           
                      - color                                                                                                                            
                      - box_color                                                                                                                        
                      - width                                                                                                                            
                      Notes: with a simple list by default it will return the value in the list - if you want the menu number then use rtrn='k' option!!!
                      returns: either key(s) or value(s) as a string or list (if multiple), your choice                                                  
                      examples:                                                                                                                          
                      > tsts = [{1: "one", 2: "two", 3: "three"},["one", "two", "three"], {"file1": "path/to/file1", "file2": "path/to/file2"} ]         
                      > for tst in tsts:                                                                                                                 
                      ...     ans = gselect(tst, rtrn="v")                                                                                               
                      ...     ans = gselect(tst, rtrn="k")                                                                                               
--------------
gtable:               purpose: returns lines or displays a colorized table from a list_of_lists, df, or dictionary                    
                      input: rows: str|list|dict|list_of_lists|list_of_dicts|dataframe                                                
                      options:                                                                                                        
                      - color: str,                                                                                                   
                      - box_style: 'single', 'double', 'solid',                                                                       
                      - box_color: str,                                                                                               
                      - header: bool_val,  # header | hdr                                                                             
                      - colnames: list | str,  'firstrow' | 'firstline' | 'keys'                                                      
                      - col_colors: list,   # gtable will use this list of colors to set each column, repeats if more cols than colors
                      - neg: bool_val,                                                                                                
                      - alt: bool_val,                                                                                                
                      - alt_color: str,                                                                                               
                      - title: str,                                                                                                   
                      - footer: str,                                                                                                  
                      - indexes: bool,                                                                                                
                      - box_style: str,                                                                                               
                      - max_col_len|col_limit|col_len...: int,  default=60                                                            
                      - human: bool,                                                                                                  
                      - rnd: int,                                                                                                     
                      - sortby: str,                                                                                                  
                      - filterby: dict {'field': 'contains'}                                                                          
                      - select_cols: list    # specify which columns to include - table will be in same order                         
                      - excluded_cols: list  # specify which columns to exclude                                                       
                      - write_csv: str,                                                                                               
                      - skip: bool           # tells gtable to skip lines of the wrong length - be careful w/this                     
                      - cell_pad=' ': str    # you can set the padding char(s)                                                        
                      - strip: bool          # strip all white space off of ever element in every row                                 
                      returns lines: list                                                                                             
                      Notes:                                                                                                          
                      if colnames="firstrow" then the firstrow will be extracted and used for the header                              
                      if colnames="keys" and we are passed a dictionary then the colnames will be the dictionary keys                 
                      I frequenly use this function for financial data analysis or csv files                                          
                      TODO: add head: int and tail: in                                                                                
--------------
handleCFG:            purpose:  if no cfg_file given it will find the default and return cfg_d (dictionary of dictioanries: cfg.sections; cfg.elem:vals)
                      input: cfg_file: str                                                                                                              
                      defaults: cfg_file if it exists is: {myappname.basename}.cfg                                                                      
                      returns: cfg_d: dict (dictionary of dictionaries - cfg.sections with key, val pairs)                                              
                      use:                                                                                                                              
                      cfg_d = handleCFG("/my/path/to/myapp.cfg")                                                                                        
                      title = cfg_d['menu']['title']                                                                                                    
--------------
handleOPTS:           Usage:                                                   
                      gtools.py [-hEP] [--dir] [<filename>]                    
                      gtools.py -s <func>                                      
                      gtools.py -S <func>                                      
                      gtools.py -t [<func>]                                    
                      gtools.py -T <func> [<fargs>...]                         
                      Options                                                  
                      -h                Help                                   
                      --dir             prints all the functions here and exits
                      -t                runs all doctest calls                 
                      -T <func>         runs specified func                    
                      -v, --version     Prints version                         
                      -s <func>         Show code block                        
                      -S <func>         Show __doc__                           
--------------
has_alnum:            Is a string blank or all white space...
                      isspace() does the samething           
--------------
ireplace:             purpose: index replace                                              
                      To get indices use eg:                                              
                      iter = re.finditer(rf"{c}", s)                                      
                      indices = [m.start(0) for m in iter]                                
                      # next line removes first two and last two indices - just an example
                      indices = indices[2:-2]                                             
                      then use this func with:                                            
                      s: is the string to do the replacements                             
                      indices: list of indexed positions                                  
                      char: is the char to replace with                                   
--------------
isnumber:             purpose: determines if x is a number even if it is a percent, or negative, or is 2k or 4b or 10M etc                                            
                      input: x: str|float|int                                                                                                                         
                      returns: True|False                                                                                                                             
                      notes: tests... pos, neg, floats, int, scientific, B(illion), T(trillions), G(ig.*) Kb(ytes|its), Mb(ytes)                                      
                      Can be used on financial data which often includes M(illions) or B(illions)                                                                     
                      In tables I use this to decide if "x" should get right justified                                                                                
                      >>> nums = [0.00, "0.00", "1.2", "+1.2", "-1.2", "1.2B", " 1.2M ", "1.2Kb ", "1.2Mb", "1.2e-9", "1.2%", 42.25, 1.1116174459457397, "2022-01-01"]
                      >>> for num in nums:                                                                                                                            
                      ...     isnumber(num)                                                                                                                           
                      True                                                                                                                                            
                      True                                                                                                                                            
                      True                                                                                                                                            
                      True                                                                                                                                            
                      True                                                                                                                                            
                      True                                                                                                                                            
                      True                                                                                                                                            
                      True                                                                                                                                            
                      True                                                                                                                                            
                      True                                                                                                                                            
                      True                                                                                                                                            
                      True                                                                                                                                            
                      True                                                                                                                                            
                      False                                                                                                                                           
--------------
key_swap:             purpose: switch or change the keyname on an element in a dictionary
                      args:                                                              
                      orig_key  # original key name                                      
                      new_keyA  # new key name                                           
                      d         # dictionay to change                                    
                      returns: the altered dictionary                                    
--------------
kv_cols:              input: my_d: dict cols:default=3 <-- both args ie: dict and cols are required!                      
                      options:                                                                                            
                      - title, header, pad, box_style, box_color: str, color: str, neg: bool,                             
                      - prnt: bool,  footer,title: str, rjust_cols: list, sep,pad: str, max_col_width: int,               
                      - centered: bool, box_style: str, human: bool, rnd: bool, box_title: bool (requires title), sep: str
                      returns: lines tabalized key-value pairs                                                            
--------------
kvarg_val:            purpose: returns a value when the key in a key=value pair matches any key given                                                               
                      NOTE: key can be a string or a list of strings                                                                                                
                      option: dflt="Whatever default value you want"                                                                                                
                      use: used in function to get a value from a kwargs ky=value pair                                                                              
                      - eg:                                                                                                                                         
                      def my_function(*args, **kwargs):                                                                                                             
                      txt_center = kvarg_val(["text_center", "txt_cntr", "txtcntr"], kwargs, dflt=1)                                                                
                      - so if you call my_function(txt_center=99) then txt_center will be set to 99                                                                 
                      ---                                                                                                                                           
                      If any key in the list is set = to a value, that value is returned                                                                            
                      see: bool_val which process both args and kvargs and returns bool_val                                                                         
                      input key(string), kvargs_d(dictionary of key,vals), default(string; optional)                                                                
                      purpose: return a value given by a key=value pair using a matching key (in key list if desired)                                               
                      options:                                                                                                                                      
                      -  key provided can be a string or a list of strings                                                                                          
                      -  dflt="Whatever default value you want - can be a string, list, int, float... whatever" <-- this is optional, if not declared "" is returned
                      if key in kvargs:                                                                                                                             
                      return val                                                                                                                                    
                      else:                                                                                                                                         
                      return default                                                                                                                                
                      returns str(key_val) or default_val(which is "" if none is provided)                                                                          
--------------
lineno:               purpose: returns current line number - primarily used for debugging
--------------
list_files:           purpose: prints a list of enumerated basename filenames (sorted by name)
                      input:                                                                  
                      dirs=list|str                                                           
                      options:                                                                
                      pattern: str|list = "*"  # glob pattern                                 
                      return_msgs<bolean> = False                                             
                      prnt<bolean> = False                                                    
                      dirs: bool               # include dirs                                 
                      dir_only: bool           # only dirs                                    
                      links: bool              # include links                                
                      returns:                                                                
                      a sorted list of those names                                            
                      or                                                                      
                      return_msgs and sorted names                                            
                      use: list_files("/tmp")                                                 
--------------
DEBUG: [gtools.py:get_mod_docs:8191] func: long_proc:             e: 'NoneType' object has no attribute 'split'
main:                 purpose: allows user to see some of the fuctionality of this tool set
--------------
max_width_lol:        max_width for each "column" in a list of lists
                      this is a way of truncating "columns"         
                      need more info here                           
--------------
maxof:                purpose: returns length of longest member of a list (escape_ansi)
                      saves me from having to look up how to do this all the time      
--------------
nclen:                purpose: finds the length of a string minus any ansi-codes and returns that lenght
                      returns: length                                                                   
                      no color len of line (length w/o ansii codes)                                     
                      if 'rich' in args then stip rich color code                                       
--------------
path_to:              Check whether `name` is on PATH and marked as executable.
--------------
pp_d:                 purpose: pretty print a dictionary      
                      deprecated as gtable does this          
                      with between (str) placed between elems 
                      and kv_s (str) between k and v          
                      >>> d = {"one": 1, "two": 2, "three": 3}
                      >>> print(pp_d(d))                      
                      one: 1  two: 2  three: 3                
--------------
do_prcnt_bar:         purpose: displays a percentge bar                                                                                          
                      args: amnt (prcnt)                                                                                                         
                      options: full_range=100 if you submit this prcnt will be based on it                                                       
                      - bar_width=40: int   # declares width of bar                                                                              
                      - color: str          # text color                                                                                         
                      - done_color:         # color of done portion                                                                              
                      - undone_color:       # color of undone portion                                                                            
                      - done_chr: str       # done character                                                                                     
                      - undone_chr: str     # undone character                                                                                   
                      - prompt: str         # prompt (before the bar)                                                                            
                      - suffix: str         # suffix (afterr the bar)                                                                            
                      - brackets=["[", "]"]: list                                                                                                
                      - show_prcnt=True: bool  # include the percentage at the end                                                               
                      - prnt=False: bool  # False to allow use in dashboards  # you can tell this to not print so you can include in in a box etc
                      returns: percent bar line as a str                                                                                         
                      #>>> rh = 56                                                                                                               
                      #>>> rl = 50                                                                                                               
                      #>>> cp = 51                                                                                                               
                      #>>> amnt = cp - rl                                                                                                        
                      #>>> full_range = rh - rl                                                                                                  
                      #>>> print(f"rl {do_prcnt_bar(amnt,full_range)} rh")                                                                       
                      # rl [██████----------------------------------]16% rh                                                                      
--------------
pretty_array:         prints out nested arrays into readable format using yaml
--------------
DEBUG: [gtools.py:get_mod_docs:8191] func: printit:               e: 'default'
progress:             # prcnt = 0.20                                                             
                      # progress(prcnt, width=60)                                                
                      Percent: [############------------------------------------------------] 20%
                      # or                                                                       
                      >>> for i in range(100):                                                   
                      ...     time.sleep(0.1)                                                    
                      ...     progress(i/100.0, width=60)                                        
                      Percent: [############################################################] 99%
--------------
purge:                removes files and dirs that match the pattern given from the location=dir
                      requires:                                                                
                      import os                                                                
                      import shutil                                                            
                      Becareful with this!                                                     
--------------
purify_file:          purpose: de-comments a file, aka removes all comments denoted by "#" ...
                      input: file: str                                                        
                      return lines: list                                                      
--------------
purify_line:          strips off and comments
--------------
pyscraper:            I may deprecate this as I rarely use it                                                                                  
                      example: pat                                                                                                             
                      # pat = '<span class="Trsdu\(0.3s\) Fw\(b\) Fz\(36px\) Mb\(-4px\) D\(ib\)" data-reactid=".*?".*?>.*?</span.*?>'   # noqa:
                      Be careful... pay attention to the html page when using a script ... many sites detect the script and block real output  
                      Required:                                                                                                                
                      from urllib.request import urlopen                                                                                       
                      import re                                                                                                                
--------------
quick_plot:           purpose: quick display of data in  a file or in dataframe                                                                       
                      displays a plot on a web browser if requested                                                                                   
                      args: data: df | str(filename: csv or dat)                                                                                      
                      options: show: bool, choose: bool, footer: str, footer: str tail: int (for the last n rows of the df)                           
                      choose invokes gselect multi mode to allow selections of columns to display in the plot (graph)                                 
                      tail, title and footer only affect the gtable if show is True                                                                   
                      return: df                                                                                                                      
                      NOTE: if a filename is used as data it will get "purified" by removing all comments first (except the first line of a dat file.)
--------------
reduce_line:          reduce a line to no more than max_len with and no broken words      
                      then return the reduced_line, and remaining_line                    
                      Note - use textwrap for this now -- reduce_line should be depracated
--------------
regex_col:            regex for a patern in a word|column                          
                      file_lines can be a filename or lines (list)                 
                      col starts at 0 to be consistent with coding standards       
                      returns lines where pat matches col number word ie words[col]
--------------
remap_keys:           purpose: remaps keys names AND can select only the k.v pairs you want (ie option: 'mapped_only')                 
                      options:                                                                                                         
                      mapped_only: bool,                                                                                               
                      rnd: int # rounds out numbers to rnd scale                                                                       
                      returns: my_d (remapped and optionally selected pairs)                                                           
                      notes: remap_d should be dict {orig_key: new_key, ...} but can be a list only (assumes and sets mapped_only=True)
                      created: 20220423 gwm                                                                                            
--------------
replace_all:          replaces ever dict key with dict value in a string
                      with_d eg: {'	', '  ', 'foo', 'bar'}              
                      TODO need more doc info here                      
--------------
retry:                there is a module called retrying that deserves more research and it provides a wrapper function called @retry()
                      use:                                                                                                            
                      @retry(5, MySQLdb.Error, timeout=0.5)                                                                           
                      def the_db_func():                                                                                              
                      # [...]                                                                                                         
                      pass                                                                                                            
                      untested - unused - completely expimental                                                                       
                      the same as                                                                                                     
                      for attempts in range(3):                                                                                       
                      try:                                                                                                            
                      do_work()                                                                                                       
                      break                                                                                                           
                      except Exception as e:                                                                                          
                      print(f"Attempts: {attempts}. We broke with error: {e}")                                                        
--------------
rgb:                  purpose: translated rgb(r,g,b) text into ansi color CODE    
                      input: r, g, b, text                                        
                      prfx: bool = False                                          
                      bg: bool = False # if set to true the color is applied to bg
                      returns: rgb color coded text                               
--------------
rootname:             purpose: returns the root name of a full filename (not path, no extension)
                      input: filename: str                                                      
                      returns: ROOT_NAME: str                                                   
                      >>> print(rootname(__file__))                                             
                      gtools3                                                                   
--------------
ruleit:               This is for development purposes                                     
                      It draws a rule across the screen and that is all it does            
                      It fails to prepare your meals or schedule your week's agenda, sorry.
--------------
run_cmd:              purpose: runs cmd and returns output                                 
                      eg: out = run_cmd("uname -o",False)                                  
                      # now you can print the output from the cmd:                         
                      print(f"out:{out}")                                                  
                      options:                                                             
                      - lst: bool   # ouput will return as a list of line rather than a str
                      - rc: bool    # returns the cmd return code instead of output        
                      - runas: str  # you can declare who to run the cmd as                
                      returns: output from command                                         
                      Note: if runas == sudo then the command will be sun with sudo...     
                      -  this function strips out all ansi code and filters all errors     
                      Test:                                                                
                      >>> r = run_cmd("uname -o")                                          
                      >>> print(r)                                                         
                      GNU/Linux                                                            
                      <BLANKLINE>                                                          
--------------
run_cmd_threaded:     purpose: runs a cmd as a thread                                               
                      options:                                                                      
                      - lst: bool  # returned output lines will be put into a list rather than a str
                      return: output from cmd                                                       
                      Note: Please, be aware that a result will be returned only after this finishes
                      so put it "later" rather than "sooner" in your app                            
--------------
sayit:                purpose: will use computer voice to "say" the msg
                      options                                          
                      - prnt: will print as well                       
                      returns: None                                    
--------------
select_file:          purpose: select a file (or dir) from using pattern(s)
                      required:                                            
                      options:                                             
                      - path: str|list  (defaults to "./")                 
                      - pattern: str|list                                  
                      - prompt: str                                        
                      - mtime: bool      # include mtime in listing        
                      - centered                                           
                      - dirs: bool       # include dirs                    
                      - dirs_only: bool  # only directories                
                      - prnt: bool                                         
                      - shadow                                             
                      - footer                                             
                      - width=0                                            
                      use: f = select_file("/home/user","*.txt")           
                      prints a file list and then asks for a choice        
                      returns basename of the filename selected            
                      Note: this uses list_files                           
--------------
shades:               purpose: returns a list of increasing intensity color shades               
                      requires = color: str                                                      
                      options:                                                                   
                      -   num=16  <-- number to divide into 255 ~ the number of color intensities
                      -   rtrn???  TODO return codes or text... CODES | strings/text/txt         
                      returns list # of ncreasing colors                                         
--------------
shadowed:             purpose: adds shadowing typically to a box
                      requires:                                 
                      input: lines as a list                    
                      output: lines as a list                   
                      Use this to see all the styles:           
                      msg = "this is                            
                      my message"                               
                      for n in range(0,5):                      
                      printit(centered(shadowed(boxed(msg + f"  
                      style: {n}"),style=n)))                   
--------------
sorted_add:           purpose: to insert a line between two patterns in a sorted way                                     
                      20210531 WIP!                                                                                      
                      TODO if after and before are empty just use the whole file                                         
                      purpose: insert line into filename between after and before patterns                               
                      the patterns need to be regex ie: r"pattern"                                                       
                      assumes the block from after to before is sorted                                                   
                      returns new_lines                                                                                  
                      eg:                                                                                                
                      >>> line = 'Insert this line (alphabetically) after "^-alt" but before "^-[a-zA-Z0-9] within block'
                      >>> filename = "/home/geoffm/t.f"                                                                  
                      >>> after = r"^-alt"                                                                               
                      >>> before = r"^-[a-zA-Z0-9]"                                                                      
                      >>> lines = sorted_add(filename, line, after, before)                                              
                      >>> printit(lines)                                                                                 
--------------
split_codes:          purpose to split out ansi codes and return them                                                                   
                      - used in color_neg()                                                                                             
                      input: elem: str (that contains ansi codes for color)                                                             
                      options: TODO include elem dflt=False                                                                             
                      returns: codes: list (unless elem=True, then it is a dictionary with preffix, elem, and suffix as key/value pairs)
--------------
sub_color:            purpose: substiture ansi color CODE for given color
                      returns ansi-CODE                                  
--------------
transcript_start:     Start class Transcript(object=filename),
                      appending print output to given filename
--------------
transcript_stop:      Stop class Transcript() and return print functionality to normal
--------------
try_it:               BROKEN BROKEN BROKEN                                               
                      This is a wrapper function for running any function that might fail
                      - this will report the error and move on                           
                      use:                                                               
                      @try_it                                                            
                      def my_func():                                                     
                      print("if this failed an error would be reported ")                
                      my_func()                                                          
--------------
try_to_get:           may deprecate this as I rarely use it                           
                      This is for use with selenium                                   
                      requires:                                                       
                      from selenium import webdriver                                  
                      from selenium.webdriver.common.keys import Keys                 
                      from selenium.webdriver.common.by import By                     
                      from selenium.webdriver.support.ui import WebDriverWait         
                      from selenium.webdriver.support import expected_conditions as EC
--------------
usr_update:           purpose: given a dict, and a list of keys to change - allow user update(s) to values in  my_d dictionary
                      go through the list of keys to fix and prompt user for new value, present the current value as default  
                      args: my_d: dict    # dict to have updated                                                              
                      fix_l=[]: list   # list of keys to prompt user for change; if empty use all the my_d keys               
                      returns: my_d (with user updates)                                                                       
                      NOTE: what is in the passed dict values is what will be presented as the default.                       
--------------
v4k:                  purpose: return a value form a dictionary (my_d) for a given key (k)
                      return value for key in dict ie v for my_d[k]                       
                      returns None is not found                                           
--------------
wrapit:               input is sentence which can be a string or list              
                      returns lines list wrapped using length and color if provided
                      NOTE: all color codes will get stripped out before processing
--------------
xlate_clr:            purpose: translates special color names to rgb(r,g,b) for processing                                  
                      requires: special color eg:                                                                           
                      - greyXX | grayXX  # where XX is a precent gradient of gray from 0,0,0 for black! to 255,255,255 white
                      - white!           # solid 255,255,255 white                                                          
                      - black!           # solid 0,0,0 black                                                                
                      - red!             # solid bright 255,0,0 red                                                         
                      - green!           # solid bright 0,255,0 green                                                       
                      - blue!            # solid bright 0,0,255 blue                                                        
                      - blue!            # solid bright 0,0,255 blue                                                        
                      - blue!            # solid bright 0,0,255 blue                                                        
                      - yellow!          # solid bright 255,255,0 yellow                                                    
                      - magenta!         # solid bright 255,0,255 magenta                                                   
                      - cyan!            # solid bright 0,255,255 cyan                                                      
                      returns: rgb(r,g,b) text string instead of supplied color                                             
                      --== Xlate special colors to rgb() ==--                                                               
--------------

Geoff McNamara

"Do not meddle in the affairs of wizards, for they are subtle and quick to anger.” J.R.R Tolkien

Elizabeth City, NC https://www.companionway.net



Credits: