Skip to main content

formfyxer.pdf_wrangling

annotations​

io​

math​

re​

statistics​

Enum​

tempfile​

copy​

Any​

Callable​

Dict​

Iterable​

Optional​

List​

Union​

Tuple​

BinaryIO​

Mapping​

TypedDict​

cast​

Sequence​

Path​

random​

cv2​

config​

get_checkboxes​

np​

convert_from_path​

pikepdf​

Pdf​

canvas​

magenta​

pink​

blue​

PDFLayoutAnalyzer​

TextConverter​

LAParams​

LTPage​

LTTextBoxHorizontal​

LTChar​

LTContainer​

LTAnno​

LTText​

LTTextBox​

LTTextBoxVertical​

LTTextGroup​

LTTextLine​

LTImage​

LTItem​

PDFUnicodeNotDefined​

PDFDocument​

PDFResourceManager​

PDFPageInterpreter​

PDFPage​

PDFParser​

PDFDevice​

resolve1​

PSLiteral​

PSKeyword​

decode_text​

translate_matrix​

mult_matrix​

MATRIX_IDENTITY​

DEBUG​

FieldType Objects​

class FieldType(Enum)

TEXT​

Text input Field

AREA​

Text input Field, but an area

SIGNATURE​

CHECK_BOX​

LIST_BOX​

allows multiple selection

CHOICE​

allows only one selection

RADIO​

__str__​

def __str__() -> str

PikeField Objects​

class PikeField(TypedDict)

type: str​

var_name: str​

all: pikepdf.objects.Object​

BoundingBox​

BoundingBoxF​

XYPair​

_to_int_bbox​

def _to_int_bbox(bbox: BoundingBoxF) -> BoundingBox

FormField Objects​

class FormField()

A data holding class, used to easily specify how a PDF form field should be created.

__init__​

def __init__(field_name: str,
             type_name: Union[FieldType, str],
             x: int,
             y: int,
             font_size: Optional[int] = None,
             tooltip: str = "",
             configs: Optional[Dict[str, Any]] = None)

Constructor

Arguments​

  • x - the x position of the lower left corner of the field. Should be in X,Y coordinates, where (0, 0) is the lower left of the page, x goes to the right, and units are in points (1/72th of an inch)
  • y - the y position of the lower left corner of the field. Should be in X,Y coordinates, where (0, 0) is the lower left of the page, y goes up, and units are in points (1/72th of an inch)
  • config - a dictionary containing any keyword argument to the reportlab field functions, which will vary depending on what type of field this is. See section 4.7 of the reportlab User Guide
  • field_name - the name of the field, exposed to via most APIs. Not the tooltip, but users1_name__0

make_textbox​

@classmethod
def make_textbox(cls, label: str, field_bbox: BoundingBox,
                 font_size: int) -> "FormField"

make_textarea​

@classmethod
def make_textarea(cls, label: str, field_bbox: BoundingBox,
                  font_size: int) -> "FormField"

make_checkbox​

@classmethod
def make_checkbox(cls, label: str, bbox: BoundingBox) -> "FormField"

from_pikefield​

@classmethod
def from_pikefield(cls, pike_field: PikeField) -> "FormField"

get_bbox​

def get_bbox() -> BoundingBoxF

__str__​

def __str__()

__repr__​

def __repr__()

_create_only_fields​

def _create_only_fields(io_obj,
                        fields_per_page: Iterable[Iterable[FormField]],
                        font_name: str = "Courier",
                        font_size: int = 20)

Creates a PDF that contains only AcroForm fields. This PDF is then merged into an existing PDF to add fields to it. We're adding fields to a PDF this way because reportlab isn't able to read PDFs, but is the best feature library for writing them.

_normalize_signature_fields​

def _normalize_signature_fields(pdf: Pdf,
                                signature_field_names: Iterable[str]) -> None

Convert ReportLab text fields into PDF signature fields.

ReportLab does not expose an AcroForm signature widget API, so _create_only_fields creates signature placeholders as text fields. This function changes those generated widgets to /FT /Sig before they are copied into the destination PDF.

set_fields​

def set_fields(in_file: Union[str, Path, BinaryIO],
               out_file: Union[str, Path, BinaryIO],
               fields_per_page: Iterable[Iterable[FormField]],
               *,
               overwrite=False)

Adds fields per page to the in_file PDF, writing the new PDF to a new file.

Example usage:

set_fields('no_fields.pdf', 'four_fields_on_second_page.pdf',
  [
    [],  # nothing on the first page
    [ # Second page
      FormField('new_field', 'text', 110, 105, configs={'width': 200, 'height': 30}),
      # Choice needs value to be one of the possible options, and options to be a list of strings or tuples
      FormField('new_choices', 'choice', 110, 400, configs={'value': 'Option 1', 'options': ['Option 1', 'Option 2']}),
      # Radios need to have the same name, with different values
      FormField('new_radio1', 'radio', 110, 600, configs={'value': 'option a'}),
      FormField('new_radio1', 'radio', 110, 500, configs={'value': 'option b'})
    ]
  ]
)

Arguments​

  • in_file - the input file name or path of a PDF that we're adding the fields to
  • out_file - the output file name or path where the new version of in_file will be written. Doesn't need to exist.
  • fields_per_page - for each page, a series of fields that should be added to that page.
  • owerwrite - if the input file already some fields (AcroForm fields specifically) and this value is true, it will erase those existing fields and just add fields_per_page. If not true and the input file has fields, this won't generate a PDF, since there isn't currently a way to merge AcroForm fields from different PDFs.

Returns​

Nothing.

rename_pdf_fields​

def rename_pdf_fields(in_file: Union[str, Path, BinaryIO],
                      out_file: Union[str, Path, BinaryIO],
                      mapping: Mapping[str, str]) -> None

Given a dictionary that maps old to new field names, rename the AcroForm field with a matching key to the specified value.

Example​

rename_pdf_fields('current.pdf', 'new_field_names.pdf',
    \{'abc123': 'user1_name', 'abc124', 'user1_address_city'\})

Args:
  in_file: the filename of an input file
  out_file: the filename of the output file. Doesn't need to exist,
      will be overwritten if it does exist.
  mapping: the python dict that maps from a current field name to the desired name

Returns:
  Nothing

<a id="formfyxer.pdf_wrangling._get_named_parent"></a>

### \_get\_named\_parent

```python
def _get_named_parent(field) -> Optional[Any]

unlock_pdf_in_place​

def unlock_pdf_in_place(in_file: Union[str, Path, BinaryIO]) -> None

Try using pikePDF to unlock the PDF it it is locked. This won't work if it has a non-zero length password.

_unnest_pdf_fields​

def _unnest_pdf_fields(field,
                       parent_name: Optional[List[str]] = None,
                       parent_type: Optional[str] = None,
                       parent_flags: Optional[int] = None) -> List[PikeField]

has_fields​

def has_fields(pdf_file: str) -> bool

Check if a PDF has at least one form field using PikePDF.

Arguments​

  • pdf_file str - The path to the PDF file.

Returns​

  • bool - True if the PDF has at least one form field, False otherwise.

get_existing_pdf_fields​

def get_existing_pdf_fields(
        in_file: Union[str, Path, BinaryIO, Pdf]) -> List[List[FormField]]

Use PikePDF to get fields from the PDF

swap_pdf_page​

def swap_pdf_page(*,
                  source_pdf: Union[str, Path, Pdf],
                  destination_pdf: Union[str, Path, Pdf],
                  source_offset: int = 0,
                  destination_offset: int = 0,
                  append_fields: bool = False,
                  anchor: bool = False) -> Pdf

(DEPRECATED: use copy_pdf_fields) Copies the AcroForm fields from one PDF to another blank PDF form. Optionally, choose a starting page for both the source and destination PDFs. By default, it will remove any existing annotations (which include form fields) in the destination PDF. If you wish to append annotations instead, specify append_fields = True

copy_pdf_fields​

def copy_pdf_fields(*,
                    source_pdf: Union[str, Path, Pdf],
                    destination_pdf: Union[str, Path, Pdf],
                    source_offset: int = 0,
                    destination_offset: int = 0,
                    append_fields: bool = False,
                    anchor: bool = False) -> Pdf

Copies the AcroForm fields from one PDF to another blank PDF form (without AcroForm fields). Useful for getting started with an updated PDF form, where the old fields are pretty close to where they should go on the new document.

Optionally, you can choose a starting page for both the source and destination PDFs. By default, it will remove any existing annotations (which include form fields) in the destination PDF. If you wish to append annotations instead, specify append_fields = True Set anchor=True to translate field rectangles based on nearby text anchors in the destination PDF (instead of copying exact coordinates).

Example​

new_pdf_with_fields = copy_pdf_fields(
    source_pdf="old_pdf.pdf",
    destination_pdf="new_pdf_with_no_fields.pdf")
new_pdf_with_fields.save("new_pdf_with_fields.pdf")

Arguments​

  • source_pdf - a file name or path to a PDF that has AcroForm fields
  • destination_pdf - a file name or path to a PDF without AcroForm fields. Existing fields will be removed.
  • source_offset - the starting page that fields will be copied from. Defaults to 0.
  • destination_offset - the starting page that fields will be copied to. Defaults to 0.
  • append_annotations - controls whether formfyxer will try to append form fields instead of overwriting. Defaults to false; when enabled may lead to undefined behavior.
  • anchor - if true, estimate field movement per page using shared text anchors.

Returns​

A pikepdf.Pdf object with new fields. If blank_pdf was a pikepdf.Pdf object, the same object is returned.

TextAnchor Objects​

class TextAnchor(TypedDict)

text: str​

center: XYPair​

PageAnchorTransform Objects​

class PageAnchorTransform(TypedDict)

scale_x: float​

scale_y: float​

shift_x: float​

shift_y: float​

matched_anchor_pairs: List[Tuple[XYPair, XYPair]]​

_normalize_anchor_text​

def _normalize_anchor_text(text: str) -> str

_textbox_center​

def _textbox_center(bbox: BoundingBoxF) -> XYPair

_extract_unique_text_anchors_from_page​

def _extract_unique_text_anchors_from_page(
        page_textboxes: List[Textbox],
        min_chars: int = 4) -> Dict[str, XYPair]

_page_size​

def _page_size(page) -> Tuple[float, float]

_estimate_page_anchor_transform​

def _estimate_page_anchor_transform(
        source_page_textboxes: List[Textbox],
        destination_page_textboxes: List[Textbox], source_page,
        destination_page) -> Optional[PageAnchorTransform]

_transform_point​

def _transform_point(point: XYPair, transform: PageAnchorTransform) -> XYPair

_local_anchor_residual_for_point​

def _local_anchor_residual_for_point(point: XYPair,
                                     transform: PageAnchorTransform,
                                     max_distance: float = 220.0) -> XYPair

_rect_center​

def _rect_center(rect: Tuple[float, float, float, float]) -> XYPair

_clamp_rect_to_page​

def _clamp_rect_to_page(rect: Tuple[float, float, float, float],
                        page) -> Tuple[float, float, float, float]

_update_annotation_rect_from_anchor_transform​

def _update_annotation_rect_from_anchor_transform(
        annotation, destination_page, transform: PageAnchorTransform) -> None

_pdf_textboxes_by_page​

def _pdf_textboxes_by_page(pdf: Pdf) -> List[List[Textbox]]

_get_page_anchor_transforms​

def _get_page_anchor_transforms(
        source_pdf: Pdf, destination_pdf: Pdf, source_offset: int,
        destination_offset: int) -> List[Optional[PageAnchorTransform]]

BoxPDFPageAggregator Objects​

class BoxPDFPageAggregator(PDFLayoutAnalyzer)

__init__​

def __init__(rsrcmgr: PDFResourceManager,
             pageno: int = 1,
             laparams: Optional[LAParams] = None)

render_char​

def render_char(matrix, font, fontsize, scaling, rise, cid, ncs, graphicstate)

receive_layout​

def receive_layout(ltpage: LTPage) -> None

get_result​

def get_result() -> List[LTPage]

BracketPDFPageAggregator Objects​

class BracketPDFPageAggregator(PDFLayoutAnalyzer)

__init__​

def __init__(rsrcmgr: PDFResourceManager,
             pageno: int = 1,
             laparams: Optional[LAParams] = None)

render_char​

def render_char(matrix, font, fontsize, scaling, rise, cid, ncs, graphicstate)

receive_layout​

def receive_layout(ltpage: LTPage) -> None

get_result​

def get_result() -> List[LTPage]

JinjaFieldTextConverter Objects​

class JinjaFieldTextConverter(TextConverter)

render_char​

def render_char(matrix, font, fontsize: float, scaling: float, rise: float,
                cid: int, ncs, graphicstate) -> float

PDFPageAndFieldInterpreter Objects​

class PDFPageAndFieldInterpreter(PDFPageInterpreter)

__init__​

def __init__(rsrcmgr: PDFResourceManager, device: PDFDevice, doc) -> None

dup​

def dup() -> "PDFPageInterpreter"

get_fields_on_page​

def get_fields_on_page(page_id)

process_page​

def process_page(page) -> None

get_original_text_with_fields​

def get_original_text_with_fields(input_file, output_file)

Gets the original text of the document, with the names of the fields in jinja format ({{field_name}})

TextAndFieldConverter Objects​

class TextAndFieldConverter(TextConverter)

receive_layout​

def receive_layout(ltpage: LTPage) -> None

Textbox Objects​

class Textbox(TypedDict)

textbox: LTTextBoxHorizontal​

bbox: BoundingBoxF​

_get_nested_textboxes​

def _get_nested_textboxes(obj)

get_textboxes_in_pdf​

def get_textboxes_in_pdf(in_file: Union[str, Path, BinaryIO],
                         line_margin=0.02,
                         char_margin=2.0) -> List[List[Textbox]]

Gets all of the text boxes found by pdfminer in a PDF, as well as their bounding boxes

get_bracket_chars_in_pdf​

def get_bracket_chars_in_pdf(in_file: Union[str, Path, BinaryIO],
                             line_margin=0.02,
                             char_margin=0.0) -> List

Gets all of the bracket characters ('[' and ']') found by pdfminer in a PDF, as well as their bounding boxes TODO: Will eventually be used to find [ ] as checkboxes, but right now we can't tell the difference between [ ] and [i]. This simply gets all of the brackets, and the characters of [hi] in a PDF and [ ] are the exact same distance apart. Currently going with just "[hi]" doesn't happen, let's hope that assumption holds.

pts_in_inch​

dpi​

unit_convert​

def unit_convert(pix)

img2pdf_coords​

def img2pdf_coords(img, max_height)

intersect_bbox​

def intersect_bbox(bbox_a, bbox_b, vert_dilation=2, horiz_dilation=2) -> bool

bboxes are [left edge, bottom edge, horizontal length, vertical length]

intersect_bboxs​

def intersect_bboxs(bbox_a,
                    bboxes,
                    vert_dilation=2,
                    horiz_dilation=2) -> Iterable[bool]

Returns an iterable of booleans, one of each of the input bboxes, true if it collides with bbox_a

contain_boxes​

def contain_boxes(bbox_a: BoundingBoxF, bbox_b: BoundingBoxF) -> BoundingBoxF

Given two bounding boxes, return a single bounding box that contains both of them.

get_dist_sq​

def get_dist_sq(point_a: XYPair, point_b: XYPair) -> float

returns the distance squared between two points. Faster than the true euclidean dist

get_dist​

def get_dist(point_a: XYPair, point_b: XYPair) -> float

euclidean (L^2 norm) distance between two points

_default_text_field_name​

def _default_text_field_name(page_index: int, field_index: int,
                             field_bbox: BoundingBox, font_size: int) -> str

_default_checkbox_name​

def _default_checkbox_name(page_index: int, field_index: int,
                           field_bbox: BoundingBox) -> str

_closest_textbox​

def _closest_textbox(field_bbox: BoundingBoxF,
                     textboxes: List[Textbox]) -> Optional[Textbox]

_sanitize_field_label​

def _sanitize_field_label(text: str) -> str

_default_label_for_field​

def _default_label_for_field(field_info: FormField, textboxes: List[Textbox],
                             used_field_names: set[str]) -> Optional[str]

_next_preferred_name​

def _next_preferred_name(
        preferred_names: Optional[Sequence[str]],
        next_name_index: int,
        used_field_names: Optional[set[str]] = None
) -> Tuple[Optional[str], int]

_is_blank_text_field​

Callable[[np.ndarray, BoundingBox, int, List[Textbox]], bool] ] = None)

def _is_blank_text_field(
    img_bin: np.ndarray,
    bbox: BoundingBox,
    line_height: int,
    text_lines: List[Textbox],
    is_blank_text_field: Optional[Callable[
        [np.ndarray, BoundingBox, int, List[Textbox]], bool]] = None
) -> bool

get_connected_edges​

def get_connected_edges(point: XYPair, point_list: Sequence)

point list is always ordered clockwise from the bottom left, i.e. bottom left, top left, top right, bottom right

bbox_distance​

def bbox_distance(
    bbox_a: BoundingBoxF, bbox_b: BoundingBoxF
) -> Tuple[float, Tuple[XYPair, XYPair], Tuple[XYPair, XYPair]]

Gets our specific "distance measure" between two different bounding boxes. This distance is roughly the sum of the horizontal and vertical difference in alignment of the closest shared field-bounding box edge. We are trying to find which, given a list of text boxes around a field, is the most likely to be the actual text label for the PDF field.

bboxes are 4 floats, x, y, width and height

get_possible_fields​

Callable[[np.ndarray, BoundingBox, int, List[Textbox]], bool] ] = None)

def get_possible_fields(
    in_pdf_file: Union[str, Path],
    textboxes: Optional[List[List[Textbox]]] = None,
    preferred_names: Optional[Sequence[str]] = None,
    is_blank_text_field: Optional[Callable[
        [np.ndarray, BoundingBox, int, List[Textbox]], bool]] = None
) -> List[List[FormField]]

Given an input PDF, runs a series of heuristics to predict where there might be places for user enterable information (i.e. PDF fields), and returns those predictions.

Example​

fields = get_possible_fields('no_field.pdf')
print(fields[0][0])
# Type: FieldType.TEXT, Name: page_0_field_0, tooltip: , X: 67.68, Y: 666.0, font_size: 20, Configs: {'fieldFlags': 'doNotScroll', 'width': 239.4, 'height': 16}
# Run improve_names_with_surrounding_text(...) afterwards to derive labels from nearby text.

Arguments​

  • in_pdf_file - the input PDF
  • textboxes optional - the location of various lines of text in the PDF. If not given, will be calculated automatically. This allows us to pass through expensive info to calculate through several functions.
  • preferred_names optional - field names to assign in order as fields are detected.
  • is_blank_text_field optional - override the blank-space heuristic used for detecting text entry lines.

Returns​

For each page in the input PDF, a list of predicted form fields

ImproveNameVisitor Objects​

class ImproveNameVisitor()

__init__​

def __init__()

improve_name_with_surrounding_text​

def improve_name_with_surrounding_text(field_info: FormField,
                                       textboxes: List[Textbox]) -> FormField

AllCloseTextVisitor Objects​

class AllCloseTextVisitor()

__init__​

def __init__()

all_close_text​

def all_close_text(field_info, textboxes) -> FormField

LowestVertVisitor Objects​

class LowestVertVisitor()

Gets just the closest text to the field, and returns that

__init__​

def __init__()

lowest_vert​

def lowest_vert(fi: FormField, tbs: List[Textbox]) -> FormField

replace_in_original​

def replace_in_original(original_text, field_map)

Given the original text of a PDF (extract_text(...)), adds the field's names in their best places. Doesn't always work, especially with duplicate text.

improve_names_with_surrounding_text​

def improve_names_with_surrounding_text(
        fields: List[List[FormField]],
        textboxes: List[List[Textbox]],
        preferred_names: Optional[Sequence[str]] = None
) -> List[List[FormField]]

surrounding_text_traverse​

def surrounding_text_traverse(fields: List[List[FormField]],
                              textboxes: List[List[Textbox]],
                              visitor: Callable) -> List[List[FormField]]

get_possible_checkboxes​

def get_possible_checkboxes(img: Union[str, cv2.Mat],
                            find_small=False) -> Union[np.ndarray, List]

Uses boxdetect library to determine if there are checkboxes on an image of a PDF page. Assumes the checkbox is square.

find_small: if true, finds smaller checkboxes. Sometimes will "find" a checkbox in letters, like O and D, if the font is too small

get_possible_radios​

def get_possible_radios(img: Union[str, BinaryIO, cv2.Mat])

Even though it's called "radios", it just gets things shaped like circles, not doing any semantic analysis yet.

get_possible_text_fields​

Callable[[np.ndarray, BoundingBox, int, List[Textbox]], bool] ] = None)

def get_possible_text_fields(
    img: Union[str, BinaryIO, cv2.Mat],
    text_lines: List[Textbox],
    default_line_height: int = 44,
    is_blank_text_field: Optional[Callable[
        [np.ndarray, BoundingBox, int, List[Textbox]], bool]] = None
) -> List[Tuple[BoundingBox, int]]

Uses openCV to attempt to find places where a PDF could expect an input text field.

Caveats so far: only considers straight, normal horizonal lines that don't touch any vertical lines as fields Won't find field inputs as boxes

default_line_height: the default height (16 pt), in pixels (at 200 dpi), which is 45 is_blank_text_field: optional override for blank-space detection.

auto_add_fields​

Callable[[np.ndarray, BoundingBox, int, List[Textbox]], bool] ] = None)

def auto_add_fields(in_pdf_file: Union[str, Path],
                    out_pdf_file: Union[str, Path],
                    preferred_names: Optional[Sequence[str]] = None,
                    is_blank_text_field: Optional[
                        Callable[[np.ndarray, BoundingBox, int, List[Textbox]],
                                 bool]] = None)

Uses get_possible_fields and set_fields to automatically add new detected fields to an input PDF.

Example​

auto_add_fields('no_fields.pdf', 'newly_added_fields.pdf')

Arguments​

  • in_pdf_file - the input file name or path of the PDF where we'll try to find possible fields
  • out_pdf_file - the output file name or path of the PDF where a new version of in_pdf_file will be stored, with the new fields. Doesn't need to existing, but if a file does exist at that filename, it will be overwritten.
  • preferred_names - optional field names to assign in order.
  • is_blank_text_field - optional override for blank-space detection.

Returns​

Nothing

auto_rename_fields​

def auto_rename_fields(in_pdf_file: Union[str, Path],
                       out_pdf_file: Union[str, Path],
                       preferred_names: Optional[Sequence[str]] = None)

is_tagged​

def is_tagged(in_pdf_file: Union[str, Path, pikepdf.Pdf]) -> bool

Determines if the input PDF file is tagged for accessibility.

Arguments​

  • in_pdf_file Union[str, Path] - The path to the PDF file, as a string or a Path object.

Returns​

  • bool - True if the PDF is tagged, False otherwise.