add slice() and splice() to arrays

[bse.git] / Changes.txt

BSE 0.25 - 01 Aug 2014
========

Bug fixes:

 - user.pl (BSE::UserReg) now consistently does new-style message
   handling

 - embedding no longer messes up the variable state for article
   generation.

 - actually tokenize the !~ template expression operator.

 - audit log mailing now uses the most specific email address supplied

 - page.pl now returns a 404 if the requested page id/alias can't be found

 - the tag owner (eg. articles) tag_ids() method was returning a tag
   count rather than the tag ids

 - fix date article custom fields

 - fix product custom fields

 - don't strip '-' and '_' from linkAlias in the importer

 - ensure images have unique ids when re-orders

 - correctly validate and report on bad tag values (c46eae4)

 - use quoted-printable where needed for text parts, and always for
   html parts of mail sent via BSE::ComposeMail (08a49f2d)

 - fix subscriptions (newsletters) (6c8fac02)

 - always fallback to [shop].from for From: email address

 - fix word-wrapping for audit log emails

 - fix encoding of email content

 - select search excerpts that match the search text like the search
   engine, rather than simple text matching (d325876a)

Enhancements:

 - the article mock objects used for pregen behave more like article
   objects

 - password strength validation and account lockouts for repeated
   failed logons.

 - plain text passwords are no longer supported

 - articles/products can now be imported from CSV

 - article/product imports can now be done as "update_only" so that
   required fields aren't necessary

 - installation now uses install.cfg, which is formatted like bse.cfg

 - added a tool to hash unhashed site user and admin user passwords

 - coupons for the shop

 - global files and image no longer require an identifier

 - when transforming an article title for use in a URL, replace
   non-alphanumeric characters with '-' instead of '_'.

 - reorder.pl now allows a sort spec of 'shuffle'

 - article/global images can now have tags (a3e0dbc)

 - emails for siteusers are now whitespace validated and trimmed
   (b5fe5b6c)

 - allow add.pl to run as FastCGI (f5ea58be)

 - SVG support for article/global images

Templates:

 - add top, dynamic, generator, url variables to generated article
   templates

 - replace variables on the makeIndex html output

 - add a params variable that behaves like the tag

 - expression tags like <:= foo :> now escape as html by default.

 - added an escape() method to scalars.

 - added a shuffle() method to arrays

 - add current to the loop variable used in .for ... in

 - added convience methods image_by_name, image_by_index, file_by_name
   to article objects

 - added a set_subject() function to BSE::ComposeMail templates

 - added .while and .wrap template directives

 - allow collection_with_tags() to work on all_visible_products()

 - massive formatter re-work

 - allow defaults to be specified for .define (53c28223)

 - allow barewords for pair list keys (53c28223)

 - various preload.tmpl changes

BSE 0.24 - 11 Feb 2013
========

Bug fixes:

 - clean up access control records for article when the article is
   deleted
   https://rt4.develop-help.com/Ticket/Display.html?id=1368

 - lookup the scalecache path and uri consistently, and configure them
   in the [paths] and [uri] sections.  This may break your site if you
   didn't use the old defaults.
   https://rt4.develop-help.com/Ticket/Display.html?id=1363
   https://rt4.develop-help.com/Ticket/Display.html?id=1364

 - remove the local date and money tag definitions from the
   administrative embedded catalog tags.  This will require updates to
   your embedded catalog template (admin/catalog.tmpl in base BSE).

 - the check_versions.pl code now handles file renames correctly

 - test.cfg is now handled case sensitively so the resulting
   bse-install.cfg uses the same cases as test.cfg

 - change BSE::UserReg to use dyn_response() instead of show_page() in
   the few places still using it.

 - define access rights for the pregen actions

 - update price and tier information in the cart when the user logs in

 - several template expression fixes:

   - or is now only an operator when a distinct word in template
     expressions

   - parse () groups as primary expressions in template expressions

   - fix the list() method for hashes

 - moved [basic].public_files to [uri].public files.  RT #1359.

 - fixed example in the SYNOPSIS for BSE::Cache::Memcached

Enhancements:

 - add support for exporting report results as CSV

 - add .iterateover directive to templates

 - add a web UI to the importer

 - added documentation to BSE::UserReg, BSE::Request::Base (the
   request object)

 - template expression enhancements:

   - added several new scalar methods to template expressions

   - added the set() method to hashes in template expressions

   - added the expand() method to lists template expressions

 - BSE now searches for shipping modules in the library_path.

 - article custom fields can now be configured in [article custom
   fields] etc

 - the shopping cart is now visible to the new tag system

 - flash notices when the user manipulates the cart

 - unlisted, un-released, expired and unlinked pages are now treated
   as unavailable:

   - static content is remove if present

   - dynamic access returns a 404 error

 - added FixedIntl shipping module 

 - updated the required modules in bse_modules.pl

Templates:

 - admin/catalog.tmpl (maybe named differently) - removed the local
   date and money tag definitions.

API:

 - admin group creation now provides reasonable defaults

Testing:

 - many internal test code changes

 - update the regen_known_errors make target to match the new test
   file name

 - add actions test for BSE::UserReg

 - disable debug output for the eway test

BSE 0.23 - 07 Sep 2012
========

Bug fixes:

 - <:.set ... :> would treat an unknown value (ENOIMPL) as an error
   rather, instead of leaving the code in place

 - removed the unused Squirrel::ImageEditor and AdminUtil modules.

 - column names in generated queries are now quoted using the
   identifier quote returned by the database driver.

 - handle language ids of the form xx_XX.charset correctly when
   performing message lookups.

 - global images are now returned in the order specified in the
   editor.

Enchancements:

 - modify Generate::* classes to build templates by filename instead
   of reading them into memory.  This will allow caching by
   Squirrel::Template.
   https://rt4.develop-help.com/Ticket/Display.html?id=1341

 - add [basic].cache_templates_locally to configure Squirrel::Template
   to cache templates in memory.  This removes the overhead of
   (de-)serializing from/to an external cache, but may increase memory
   use.

 - tags can now depend on a tag from another category being selected
   before being displayed.

 - lookups in sql_statements are now cached.
   https://rt4.develop-help.com/Ticket/Display.html?id=1352

 - added collection_with_tags() to TagOwners (so it's usable for
   Articles and Products, etc) to allow filtering the various
   collection methods by tag.

 - added bse.articles and bse.products for access to the Articles and
   Products collection classes.

 - order payment would fail if the CC fields weren't present.

 - search index depth can now be configured in the config file

 - some tests now clean up the articles they create more reliably

 - $DATADIR is now configurable as [paths].data

 - $IMAGEDIR and $IMAGES_URI are now reliably configurable as
   [paths].images and [uri].images.

 - better perl 5.16 compatibility (resolved some warnings)

 - installation now updates installed scripts with the test.cfg
   configured perl.

 - tests now handle missing optional modules correctly

 - added iterateBy() to Squirrel::Table for memory efficient iteration
   over query results.

 - revamped imageclean.pl:

   - output is now controlled by templates

   - added a command-line tool

   - web UI is now access controlled

   - the web UI now provides a preview of the work to be done, with
     checkboxes to control which clean ups are done.

   - now accounts for public files controlled by BSE::TB::Files.

 - quoted shipping can now be disabled

 - added the Courier::ByUnitAU shipping module.

Infrastructure:

 - the test script directory (t) has been reorganized.

 - configuration can now be loaded from a string to simplify building
   config objects for testing.

Documentation:

 - documented [basic].all_dynamic

 - minor documentation updates to Squirrel::Table

 - added a htmldocs target to the Makefile.  This requires a
   HTMLDOCDIR parameter:

     make htmldocs HTMLDOCDIR=/somewhere

 - added tests for syntax check all pod

 - improved documentation in Articles, BSE::AdminMenu and
   Squirrel::Template::Processor.

Templates:

 - admin.pl now properly passes the admin state to bse.* template
   variables

 - added templates admin/imageclean/intro.tmpl,
   admin/imageclean/preview.tmpl and admin/imageclean/final.tmpl for
   the updated imagclean.pl

BSE 0.22 - 08 Jun 2012
========

Bug fixes:

 - the search index builder was ignoring configured field scores.

 - the customer is no longer asked to select a shipper and shipping is
   set to zero for orders where all products have zero weight.
   https://rt4.develop-help.com/Ticket/Display.html?id=1328

 - template engine: template tags implemented as a literal were no
   replaced when that literal was false. (4a446ac3)
   https://rt4.develop-help.com/Ticket/Display.html?id=1342

 - the body text markup doclink[] accepts link aliases, but didn't
   allow for "-" in the alias.  All "-" and document aliases are
   permitted.

 - shopadmin order_paid and order_unpaid are now recorded in the audit
   log instead of being added to special instructions.
   https://rt4.develop-help.com/Ticket/Display.html?id=1326

 - the scale() thumbnail operator, when used with a background would
   paste rather than rubthrough() the source, leaving tranparent areas
   black.  If a fill is supplied, it is now always used for
   transparent images.

 - generate the correct refresh URL when the user isn't logged on
   attempting to request an admin page.

Enhancements:

 - automatic image insertion can now be disabled globally or on a
   per-article basis.
   https://rt4.develop-help.com/Ticket/Display.html?id=1331

 - templating re-re-work:
   - still backward compatible
   - new, faster, more regular processing internal to tags:
     - variables supplied by code, similarly to TT, Mason
     - macro definitions
     - call macros or files with parameters and localized variables
   - integration into BSE itself still limited, but one step at a
     time.

 - briefly documented bsexlsprod.pl (site/docs/bse_import.pod)

 - bsexlsprod.pl can now update article tags

 - bsexlsprod.pl can now update tiered pricing

 - make the eimage variable available on the admin/image_edit template
   and use it to display a thumbnail.
   https://rt4.develop-help.com/Ticket/Display.html?id=1290

 - add the Courier::FixedAU shipping driver.

 - that an order was manually paid is now recorded separately from the
   payment type.
   https://rt4.develop-help.com/Ticket/Display.html?id=1325

 - the shopadmin order_paid target now accepts a paymentType variable
   to optionally set a new payment type on manual payment
   https://rt4.develop-help.com/Ticket/Display.html?id=1325

 - the shopadmin order_paid and order_unpaid now require csfrp tokens.

 - siteuser admin view and edit targets now also accept userId to
   identify the user.
   https://rt4.develop-help.com/Ticket/Display.html?id=1351

Templates:

 - admin/order_detail.tmpl - the product tag now uses tag_article as
   it should.

 - admin/order_detail.tmpl - added order, payment_types and
   payment_type_desc as template variables.

 - admin/subscr/list - isubscription is now an object tag, giving
   access to the is_removable method
   https://rt4.develop-help.com/Ticket/Display.html?id=1323

 - admin/subscr/edit, admin/subscr/detail - subscription is now an
   object tag

BSE 0.21 - 07 Mar 2012
========

 - Squirrel::Template has largely been rewritten performing a parsing
   then a processing step rather than doing many, many s/// over the
   template text.

 - verbose output of generate.pl is now based on the template
   admin/generate.tmpl

 - add the referer tag to all dynamic pages

 - added [undeletable articles] to bse.cfg as a supplement to
   @NO_DELETE.  @NO_DELETE is now deprecated and may be removed in a
   future release of BSE.
   https://rt4.develop-help.com/Ticket/Display.html?id=1209

 - iterator inlines can now be filtered (Adrian Oldham)

 - added some more extension to content type mappings for video
   formats (Adrian Oldham)

 - the shopadmin order list targets are now much more efficient - they
   no longer load the entire order table, are paginated, and let the
   data do the selection of matching records.

   NOTE: this removes some backward compatibility - iterator filters
   no longer work and a new tag, all_order_count can be used to fetch
   order counts.  Extra search parameters have been provided to filter
   the results.

 - added a new adminurl2 tag where the second parameter is a target.

 - siteusers.pl now flashes all success messages, and success message
   text is fetched from the messages table

 - added is_released and is_expired methods to Article.  This should
   be visible to article tags in templates.

 - more consistently use tag_article on the admin side of the site.

 - make tag_article smarter so it can call more methods

 - consistently use admin_tags instead of the old collection of admin
   tag methods.

 - tag_object (used for object style tags) now checks
   restricted_method() if implemented for the object.

 - the editor file iterator is now an object iterator, and the efile
   tag on the file edit page is now an object tag

 - add file_exists to BSE::TB::Article::File
   https://rt4.develop-help.com/Ticket/Display.html?id=1288

 - refactored makeIndex.pl into makeIndex.pl (web) and
   util/bse_makeindex.pl (console).
   Output from these is now encoded using the default BSE encoding.
   makeIndex.pl can now produce HTML output using admin/makeindex.tmpl
   The bse_make_index background process now uses the console version.
   https://rt4.develop-help.com/Ticket/Display.html?id=1301

 - admin-mode article links now ignore the admin value stored in the
   article record and always return a link to admin.pl for that
   article.

 - add a missing comma to BSE::UI, preventing a warning.

Bug fixes:

 - thumb.pl would return content types with doubled image/ prefixes,
   eg. image/image/jpeg

 - update BSE::AdminUsers, BSE::ChangePW, BSE::UI::AdminReport to use
   more modern admin tags
   https://rt4.develop-help.com/Ticket/Display.html?id=1234 (partial)
   (Adrian Oldham for BSE::ChangePW, BSE::UI::AdminReport)

 - with [site].secureadmin enabled, refreshes to the secure admin url
   could cause a 500 error.

 - check nomatch fields even for blank fields.  Otherwise a field with
   a LF or CR would not be caught by the dh_one_line validation rule.

 - saving a tag from the tag manager without changing the name or
   deleting a tag that doesn't exist would result in a 500 error.
   https://rt4.develop-help.com/Ticket/Display.html?id=1287

 - setting an unfilled order's stage to "shipped" now sets whoFilled
   and whenFilled for the order.
   https://rt4.develop-help.com/Ticket/Display.html?id=1286

 - an empty title (but no missing) when saving a user or group file no
   longer defaults that to the file display name.
   https://rt4.develop-help.com/Ticket/Display.html?id=1303

 - validation errors are now correctly displayed for adding a user or
   group file
   https://rt4.develop-help.com/Ticket/Display.html?id=1302

 - supplying an empty content type when saving a user or group file
   now resets the content type to that derived from the file's display
   name.
   https://rt4.develop-help.com/Ticket/Display.html?id=1304

 - fix the message id for the "move up a level to section" message in
   the possible parents drop down.

 - error_img would produce an inline stack trace from an internal
   error on a failed logon. (logon.pl)

Templates:

 - admin/order_list*.tmpl - see the order list re-work above.

 - you can now use <:- and -:> as tag delimiters to consume whitespace
   on the left and right of the tag.

 - Squirrel::Template now documents the template syntax more regularly.

 - Squirrel::Template is now much faster in most cases.

 - admin/subscr/detail - the subscription orders iterator now includes
   the billFirstName, billLastName and filled fields from the
   order. (Adrian Oldham)

Internals:

 - added a getCount() method to retrieve a count of matching records.

BSE 0.20 - 22 Dec 2011
========

Please read any NOTES below carefully.

 - store a truncated card number for credit card payments (both online
   and email).  Reinstate storage of the card holder name in ccName.
   ccNumberHash is no longer populated.

 - ifNeedDelivery (checkoutnew_base.tmpl) wasn't being reset on order
   submission.

 - the site_users table has been refactored into the bse_siteusers
   table, removing the bill* fields and adding deliv* fields.  Other
   obsolete fields have also been removed.

   NOTE: To preserve your registered users on upgrade, run the
   provided schema/site_users_to_members.sql SQL through the mysql
   tool.  This will create entries in bse_siteusers corresponding to
   those in your original site_users table.  You MUST start with an
   empty bse_siteusers table or the script will throw an error.

   NOTE: Any configuration such as report definitions, query group
   definitions should now refer to bse_siteusers instead of
   site_users.

 - Data::UUID 1.148 is now required

 - remove target entry for the shop confirm action, which was
   otherwise removed years ago. RT #1280.

 - logon now starts a new session, preserving the cart.  As a
   side-effect this logs out the admin user, though this may
   change.  RT #1279.

 - passing a new session id between the SSL and non-SSL versions of
   the site is now validated.  RT #1279.

   NOTE: This requires that [site].secret be set to a value specific
   to your site.  Running:

     openssl rand -base64 32

   generates a suitable value.

 - delete the session data for a site users session when they logoff.
   As a side effect this will log out the admin user.

 - allow dynamic content pages to display flashed messages with the
   message tag.  Note that not all actions flash a message.

Template updates:

 - checkoutpay_base.tmpl - the name of the cardHolder payment field is
   now ccName.

 - admin/order_detail.tmpl - order ccPAN can be used to display the
   truncated card number.

 - user/options_billing_base.tmpl and its generated template have been
   renamed with s/billing/delivery/ since it now contains delivery
   details.  The link to it from user/options(_base).tmpl has also
   been updated.

 - user/options_base.tmpl, user/register_base.tmpl,
   admin/users/(add,edit,view).tmpl have been updated to match the new
   site users schema.

 - user/logon_base.tmpl, user/register_base.tmpl - updated the
   "nothing is more permanent that a temporary solution" link text for
   logon, register and recover password.

 - shopitem.tmpl - display the flashed message, if any.

 - removed the unused checkoutcard and checkoutconfirm templates

 - added admin/generate.tmpl for generate.pl progress output.

 - added admin/makeindex.tmpl for search index verbose output

Internals:

 - Squirrel::Table->make now looks for default_I<colname> when looking
   for default values for a column.