Hide

Drupal 8 conversion

hide
Hide

Archived copy - no updates to be made


The conversion to Drupal 8 requires us to build a new Drupal system and the migrate our existing data into it.

Priorities are as follows:

  • C - Critical - This is required before we can effectively evaluate the migration.
  • H - High priority - Must be completed before we go live.
  • M - Migration - Data that will appear as part of the migration process in the new instance of Drupal 8 which we will build to hold the live data.
  • I - Important - Highly desirable, but if not available we can go live without it. It then needs to be prioritised.
  • U - Useful enhancement - Part of the future development plan.

See also Major unresolved issues and Other issues

Outstanding items

No. Priority Raised by Assigned to Description Comments Status
2 Useful Phil   Search and replace. Drupal 8 does not have a vbo search & replace. Other sites do not seem to have a need for this. We used it a lot as part of the initial drupal implementation and although that need has reduced would still a useful tool to have.  
11 Important Ken Ken HTML checker - needs to be ported to D8 Mainly done - a few outstanding items WIP
12 Important Ken Phil HTML errors - currently every Genuki page in D8 generates a raft of HTML errors (using the online check in the bottom right-hand corner of each screen).  Most of these errors appear to come from the Drupal system/theme itself.  We need to establish whether these are genuine HTML5 errors being reported, and if so, what do we do about them ? Phil has reduced the number of HTML errors by moving the Webform onto its own page WIP
16 Important Ken Ken Drupal 9 - need to ensure that none of the Genuki custom code contains anything that will be deprecated in D9.  The "Update Status" report will show the size of the problem    
17 Important Ken Ken Church nodes - need to ensure that the Edit Widget for the Church Location compound field has a proper Media Library widget.   WIP
19 Important Ken   Leaflet Maps - ensure there is a consistent set of map layers and map controls across the entire system.

Mostly done. What still needs attention?

See email exchange 14/3.   The debate is on need for consistency across al nodes and whether individual map layers are needed at all

We seem to have consistency across all the leaflet maps that users see apart from gazetteer nodes which have a reduced set of layers rather than the country specific sets. That task can be deferred as users are unlikely to see and use the gazetteer nodes.

The discussion about which maps individuals like and the suggestion to remove some and remove some functionality from the users is not relevant to this task

WIP
21 Important Ken  

Cron - in D7 we used Elysia Cron module to run critical cron jobs every 15 minutes.  We need to decide how we achieve this in D8 ?

Ken: Initially, we can run standard cron on an hourly basis. De-prioritising to "Important". Not started
22 Important Ken   Folder structure for media images - we need to review and possibly change the folder structure where Drupal is storing files and images.   Not started
29 Useful Peggi   I kind of focused on Longford but did wander around other areas as well.  Since I don't seem to be able to edit anything, most of my analysis covers the page(s) from a non-logged in user viewpoint.  Here's what I found:
  1. Doing a search offers an option to select a plain page - I don’t think many users will understand what that is.

1. Gareth;  Searching on Granard from Search box on a Longford place node gives results including node type, as in

GRANARD - Longford - Plain/Topic page… Granard - Longford - Gazetteer entry…

The addition of the type to the search results helps explain what it has found. If you search for Lytham it will find a few hits and the type helps you select the appropriate one. We shouldn't think of removing useful information because somebody doesn't understand all the options.

Not started
29a Useful     2.Got a Page not found error when I clicked the Churches link in the new nav

2. Gareth; Place nodes do have Churches in top topic menu but none come up, no CDB?, so link shouldn't be there ?

Page not found was fixed long ago.

But Churches still shown as a topic when there are none

An expensive search every time a user views a node even when they don't want to look for nearby churches will have a severe performance impact. So we shouldn't try to do this.

Not started
32 Useful Phil   Define a policy for auto-include as there could be a performance impact by checking for all 80 topic types before displaying a page. In d7 we just did it for gazetteers and maybe added in another couple.    
36 Important Phil Ken Church location field does not display WKT data   Not started
37 Useful Phil Ken Problems adding church icon via the location edit widget. Is it a full screen only feature?   Not started
38 Useful Phil   Find a way to easily get at media entity data within an edit screen   Not started
48 Important Phil Ken The User Errors table is appearing on lower level tabs on the My Account screen.   WIP
60 Important Brian   Drop down list of topics should only show primary topics, and should exclude sub-topics.  The dropdown list for sub-topics (currently hidden for most maintainers) should only show the sub-topics relevant to the primary topic.

The drop down list of primary topics is now limited to primary topics.  Similarly for sub-topics.  The only item outstanding would be to limit the list of sub-topics to the selected topic.  No solution identified for this part yet.

Primary tested (4/3/20 Gareth)

Part Fixed
66 Important Gareth  

Why persevere with Location Type on Gazetteer nodes when not completed  on D7 nodes ?

And Year from/to data !!  not going to happen in Wales

Not everyone will agree with these

We need and use location type, for example to identify those containing county boundaries and this is where we specify that a red line is red for these. Boundaries do change over time and we need to be able to differentiate between them. For example the county boundaries following each set of changes are available and it will help users to make them available.

Not started
67 Important Gareth   On the Node Edit screen, provide Save / Preview / Delete buttons at the top of the screen, ideally in addition to the buttons already displayed at the bottom of the screen.   Not started
79 Important Brian / Gareth Phil Currently when logging in an out of the system, the system takes you to an unexpected page. Ken: Phil has put in a temporary solution to take you to the home page.  Issue needs to be investigated further after Go Live. On hold
81 Important Gareth   Gareth has questioned whether the 'drag to reorder button' is still required against every topic section on Node Edit ? (They are now auto ordered  when adding/removing a new topic and we wouldn't want maintainers to arbitrarily change that ! - Gareth)   Not started
83 Important Gareth Ken The "What Links Here" facility in D7 is missing from D8.   Not started
84 Useful Brian   The range of map layers do not have the same range of zoom levels. If you are on a layer that does not have all the zoom levels, a blank map is shown for those zoom levels without tiles. If you are currently using a different map at a zoom level not supported by another map and try to switch to that map it does not allow you to do that.   No action possible
85 Important Brian   Bing maps not visible in test system. Likely to work in live system with no action Review once live
87 Important Phil   The vbo option to modify fields does not seem to work. Is it confused by our Topic fields? We will need this to change the Maintenance level field.   Not started
88 Important Phil   Active maps don't work on small screens such as phones as the images have been allowed to shrink. They need to be a fixed size as otherwise the mapping is wrong. Touch Devon on the England map on your phone and it takes you to Wales!  
89 Important Phil  

Editing a node which use the leaflet widget shows a php error:

User warning: The following theme is missing from the file system: leaflet_widget in drupal_get_filename() (line 295 of core/includes/bootstrap.inc).

Ken: In the early days of D8, we installed the D8 leaflet_widget module.  However, it became quickly apparent that it wasn't fit for purpose for us, and I ended up writing our own widget in a module called genuki_leaflet.

In order that the two modules didn't clash, I uninstalled and deleted the leaflet_widget module.  So, I don't really understand why there is still a PHP error referring to the uninstalled module.

It might be that, when I un-installed the leaflet_widget module, some remnant of configuration was left lying round in the system.  If this is the case, we won't get this issue when we build the D8 production system.

Alternatively, it maybe that my custom leaflet widget module is incorrectly dependent on the contributed module that I have deleted.  I will check.

 
90 Important Brian Ken When the DEV pages were loaded into D8, there are still several thousand link errors being reported, whereas in the D7 live system there are only a handful.    
91 Useful Phil   We can now add holes in the middle of areas with a boundary, but none of our mapping shows them. So we should adjust our map screens to show them and disable the click for the surrounding area within them.    

 

Completed items

  Priority Raised by Assigned to Description Comments Status
1 High Phil Ken Update migration of church pictures so that any data in the title or alt fields is moved into the media entity copyright field. In D7 we used Juicebox. to display the images. That doesn't work with media entities and so plain images are used. They can't used the Juicebox fudge for displaying the copyright info. So this has now been moved to a new field in the media entity. Completed
3 Migrate Phil Ken Migrate main menu.

All of the menu items that were created manually by Phil have been disabled.  These have been replaced by a full country / county structure.  The menu links should now work correctly for those counties that have been migrated.  For the rest, the menu links to a dummy stub entry (these can be ignored).

Work done,  Tested (11/3/20 Gareth)

Complete

4 Migrate Phil   Migrate maintenance menu to become the 'Genuki maintenance' item in the top bar of the screen for logged in users. We still need to decide where we want to display the Maintenance, Contributors, and Administration sub-menus.  Phil has proposed that, as many of these menu links are no longer used, the remaining menu links are simply added as shortcuts. Complete
5 Critical Ken Ken Ckeditor - need to ensure the D8 version of Ckeditor has all the functionality that we had in D7. eg. Media embedding, table processing, etc, etc. All the D7 table formatting has been added to D8 Ckeditor. Complete
6 High Ken Ken County boundaries - need to decide how to hold county boundaries in D8.  Options are a separate node type (as per D7), or as a type of Gazetteer entry Decision - hold county boundaries as a gazetteer entry.  Now need to implement this, and change the relevant migration scripts. Complete
7 Critical Ken Ken Media - the move to Drupal 8.8 means that the software for embedding pictures in Ckeditor fields has moved into core.  The html markup used to do this has completely changed. All text fields have now been re-imported to generate the new <drupal-media> tags Complete
8 Critical Ken Ken Feeds - need to develop D8 Feeds Importers for all Colin Hinson's files   Complete
9 High Ken Ken Errors & Statistics (Spider-2 report) - needs to be ported to D8  Done. Complete
10 High Ken Ken Link checker - needs to be ported to D8

Done

Complete
13 Important Ken Ken Other Errors checker - needs to be ported to D8. Done. Complete
14 High Ken Ken My Account page - needs to be ported to D8, to enable Maintainers to see what they are responsible for, and what errors they have outstanding. Done Complete
15   Ken Ken Email obfuscator - needs to be ported to D8. Email obfuscation module now ported from D7 to D8.    Sorted/tested (Ken) Complete
18 High Ken Ken Leaflet widget - need to implement the WKT and OS Grid Ref input fields Gazetteer locations no longer appear on edit map so cannot be changed. Complete
20 Important Ken Ken/Phil

Redundant modules - uninstall and remove all modules that we don't require:

  • paragraphs
  • field_collections
  • search (in core)
  • geophp
  • leaflet_widget
  • juicebox ?
  • embed
  • entity_embed
  • others ?
Ken: Needs thorough testing after removal from D8 Dev and/or D8 Live. Complete
23 High Ken Phil Topic nodes - in D7, Topics were a separate node type.  In D8, we have decided to incorporate these into Plain nodes.  We need to ensure that we have retained the Topic processing we had in D7.  eg. Auto-include on the relevant place node.

Done:

  • Migration script developed for D7 Topic nodes into D8 Plain nodes.

Still to do:

  • Develop/test Auto-include functionality
Completed
24 Important Ken Ken/Phil System status report - we need to resolve all the Errors (currently 2) and Warnings (currently 4). Ken: Initial work carried out by Ken on the Entity/Field Definitions error.  It is not obvious how we resolve this.  Further work required. Complete
25 High Brian Ken On the UK&I page, the The British Isles map is not active, so it does not act as a substitute for the set of six links below it Ken: On investigation, it appears that clickable maps are no longer working anywhere in the system.  Underyling issue is that Drupal have changed their way of processing embedded media (eg. maps, photos, etc), and it can't cope with the clickable map processing. Solution is to add a new Genuki version of the Media Embed output filter. Complete
26 High Brian Ken

Some (obvious) things I’ve spotted, as a user:

  1. The second and third link on the page (to maps) reach pages that have the maps missing.
  2. All the topic headings have Up Arrows. (In the current system only Chronology does.) These all lead to (i) a repeat of the topic text that has just been left, and (ii) an error report form - neither of which is I think not a good idea. I suggest all the UK&I Up arrows be removed.
  3. An empty “Churches” topic heading has been inserted.
  4. Under Dwellings there is a bibliographical citation which has an inappropriate font. (Citations use <blockquote> in D6, throughout DEV and LON, and elsewhere in BIG)
  5. Similar problem under Taxation
  6. The set of six country links have a a black dot on their left
  7. The link to England gets to the right page but opens it at the bottom - indeed this seems to apply to all the within-GENUKI links I’ve tried.

Resolutions:

  1. Required images has not been imported - fixed.
  2. Fixed
  3. Fixed
  4. Fixed
  5. Fixed
  6. Fixed - CSS issue
  7. No issue ?

Ken:  Brian has confirmed that none of the above are issues any longer, and can be closed.

Complete
27 High Gareth  

I notice that not all OSM default view maps have a button to switch to other layers.

For example, Nearby places/churches do but place nodes and church nodes don’t.

See item 19. Complete
28 High Gareth Ken Gareth and Peggi are unable to edit pages that they haven't created Ken: This is caused by the migration of permissions failing to work.  The underlying reason for the failure is due to the fact that the machine names of some of our content types have changed between D7 and D8.  Provided a mapping, and re-migrated the data. All OK now Complete
29b Useful     3.The map tab and feature is really nice.  But there are big green outlined areas on the map (Longford) but it’s not really clear to me what those are.  I read the description below the map but still don’t really understand what the green areas are showing me.  And it says “those entries linked to the same place page have a common colour.” But when I hover and get the tool tip they are two different places within Longford.  I found this a little confusing so would suggest some type of clarification.

3.David: The green shapes are three parish outlines, all are green because they all belong to County Longford. There appear to be no places on the map, I expect if any of these were there they would be indicated by coloured markers.

Have a look on the live site at https://www.genuki.org.uk/maps/county/190.

Resolved via David's comment
29c Important     4.Also, when I add a new volunteer in the current version, there are several places where changes need to be made.  I didn’t want to clutter things up by adding a fictitious maintainer so maybe there are other screens, but I didn’t see anywhere to assign them a place to manage.  Is that just something that isn’t in place yet? 4. ? Peggi to clarify - I was referring to the process we currently use wherein I have to make changes to the county - i.e. change maintainer/maintenance level, add a new user and assign them the appropriate county.  However, since I wrote this I have re-explored and found what I needed.  So this has been resolved. Resolved
30   Gareth       Deleted
31 High Phil   Define a views filter to let us choose nodes topic type within our topic field Phil: Essential to implement auto-include of plain nodes within place nodes. Completed
33 High Phil Ken genuki_html does not seem to lets us use <br> Ken: Added <br> as a valid tag to genuki_html text format Complete
34 High Phil   Add hints for field completion that will appear when editing nodes   Completed
35 High Phil Ken Try and get Imce back as a tool for managing images/uploads

Ken:  My understanding is that from Drupal 8.8 onwards, with Media embedding in Core, all new images should be uploaded by the "Insert into Media Library" icon within Ckeditor.  Therefore, no need for Imce anymore in D8 ? Doesn't seem to be used.

Imce removed by Phil

Completed
39 High Ken Ken There is no summary of errors on the My Account screen   Complete
40 High Gareth Ken Gareth is unable to load new files/images via the Ckeditor embed media button. Ken: This is a permissions issue.  All permissions for the Media module have now been created. Complete
41 High Gareth Ken Following the principle of keeping things as near to D7 as possible for maintainers, it would be a very good thing if the Ckeditor icons were in exactly the same order - they aren’t now. Ckeditor icons in D8 re-arranged to be consisten with D7 Complete
42 High Phil Ken Media fields are not being displayed correctly on Node View for PDF files (eg. Phil's Lytham PDF).

Ken: Two issues here:

- PDF files were not being migrated over from D7

- the media formatter for PDF files was showing too many fields

Complete
43 High Gareth Ken Following the loading of PER and CGN pages, the Index of Parishes pages were giving PHP errors.

The cause of this was that neither page had a "Parent Place" in D7.  Fields were completed in D7, and both nodes re-migrated.

NB. I need to ensure that the "Parent Place" field is checked for all nodes prior to migrating any further counties.

Complete
44 High Peggi / Gareth Ken Following the loading of PER and CGN pages, new countries were not available in the main menu system. Menu system manually updated in D8 to include the two new counties. Complete
45 High Gareth Ken The links produced by the dynamic content for Gazetteers give php errors.

Checked 3/3/20 seems OK now ? (Gareth)

Test OK - 4/3/20 (Ken)

Complete
46 Important Brian Ken On the Node Edit screen, huge images are rather off-putting. This is due to the responsive theme we are using.  The simple solution is to use the Adminimal admin theme for node editing.    Fixed, Tested (11/3/20 Gareth) Complete
47 High Phil Ken

On the Node Edit screen, there should be a "Remove" button to remove topics.

Ken 2/3/20 - In the absence of a "Remove topic" button, I have removed the mandatory check on the content field, which should now enable you to save the node once you have removed the topic content. NB. This will now remove that particular topic heading from the node as well.

Please note that I have left the mandatory check on the topic type field, as I cannot risk having unclassified topics saved with the node - that would create mayhem all over the place.

A contribute module found to provide a type of remove button - it display an "X" on the RHS of each topic.

Fixed - tested (4/3/20 Gareth}

Complete

49 High Gareth, David Ken The redirection facility is not working on the Broken Links report

Fixed - tested  3/3/20 (Gareth)

 

Complete

50 High Ken Ken The Church Database search (/churchdb) is not working

The initial selection screen is now working - now need to develop/migrate the underlying 2 views we have for displaying the data.

We no longer use a view to display churches on a map as that doesn't have enough functionality. We now use custom php code which does actually use a view to find the node ids to add to the map. /drupal/8-dev/web/modules/custom/genuki_pages/

src/Controller/NearbyChurchesController.php  So find the gazetteer node id and either redirect or call the subroutine in there.

Ken: a solution is now available.  Phil has provided a  further enhancement to output the results in tabular format as well as plotted on a map.

Complete
51 High Ken Phil Add "Errors & Statistics" report to Main Menu in Blue Bar   Complete
52 Important David Ken Lou Mills Belper burials page is listed under ENG, and is unformatted in D8.  Use of <pre> ?? David corrected the county code in D7.  Ken added <pre> to the list of valid HTML tags. Complete
53 High Gareth Ken All fields on the Broken Links reports are shown as field_??????. Replace field machine names with user friendly labels, including for topics.   Complete
54 High Gareth, Brian Ken On the Node Edit screen, there is a request to suppress the Sub-topic and Topic Description fields.  Neither fields are used by the majority of maintainers, and are therefore regarded as "clutter". Ideally this option should be configurable by user.   Complete
55 High Phil Ken The User Errors table is appearing on the login screen. Suppressed for non-authenticated users. Complete
56 High Ken Ken User Address field on User profile is being migrated without a Genuki HTML format. Fixed migration acript Complete
57 Critical Ken Ken In order for the D7 to D8 data migration to be able to work county by county, all nodes and all files need to have a county code assigned in D7 Ran VBO fix program Complete
58 High Brian Ken Parish maps on DEV place pages are not clickable. Fixed - tested (11/3/20 Gareth) Complete
59 Critical Ken Ken Can't save nodes when suppressing topic subfields.  An illegal choice was detected. Removed options from hidden field Complete
61 High Brian Ken On the Node Edit screen, topics should be displayed in alphabetical sequence, rather than in the order they were added. Fixed - tested (4/3/20 Gareth) Complete
62 Useful Ken Ken On the Node Edit screen, the "Add another item" should be renamed "Add another topic"   Complete
63 High Gareth Ken On the Node Edit screen, there should be a list of link errors.  Ideally this option should be configurable by user.   Complete
64 High Gareth Ken On the Node Edit screen, there should be a table of links to the topics appearing further down the page.  Ideally this option should be configurable by user.

I think it's acceptable to just show topics already in use, coupled with the Add New Topic item.

Just need anchors now (Gareth 3/3/20)  Work complete, needs testing- tested (11/3/20 Gareth)

Complete
65 High Gareth Ken

Using CKeditor to add colour to text doesn't work, inserts the code OK but it disappears with Save.  3/3/20

Fixed - tested (4/3/20 Gareth ) Complete
68 Important Ken Ken In some instances, images are not being centred on a page as they should be.  For example, http://test.genuki.uk/big/eng/DEV/Axminster/ParishMap Tested Brian 14/3/20 Complete
69 High Brian Ken Headings in DEV plain pages. The conversion has captured the font settings, but not the alignment. Many of the headings were centred (using class="gki_DEV") - these have all been lost. A class attribute has been added to all <h?> tags in the genuki_html text format.   Fixed        Tested 11/3/20 Brian Complete
70 High Brian   Many (perhaps almost all) of the DEV Plain pages (nearly eight thousand in number) start with “Not specified” - this is accurate, but annoying, as Brian was not in the habit of using the topic field in plain pages. One can in many cases easily infer what the topic should be - depending on from which topic contains the link to the plain page. But Brian can’t imagine any reasonable scheme of automating the adding the topic into all the topic field. Would it be possible simply to omit the “Not specified” rather than proclaim it?   Completed
71 High Brian Ken Some DEV images are showing, others liked to from within plain pages aren’t - instead what appears is "Unable to display image - file not migrated. See error log”.

Issue here is that some of Brian's images weren't migrated as the images in D7 had not been correctly tagged (by Ken) with the correct county code prior to migration.  Many, if not all, the files have been located in D7 and migrated over to D8. 

Fixed       Tested 11/3/20  Brian

Complete
72 Important Brian   The background to the map at the top of the England page (http://test.genuki.uk/big/eng) should be white.  It has become corrupted during the D7 to D8 migration.  The corresponding maps for Scotland, Wales, and Ireland all look ok. Seems to be fixed by accident when trying to sort out responsive images Complete
73 High Ken Ken The Genuki structure page (http://test.genuki.uk/structure) has been migrated, but is not formatted correctly. Phil: Needs the special css adding to the genuki theme Complete
74 High Brian Ken In DEV I routinely (but not invariably) ticked “Exclude title from Display” - however this directive seems to be not (yet) supported in D8. The result is not very attractive - see, e.g. http://test.genuki.uk/big/eng/DEV/Clovelly/ClovellyMystery1898. Where I didn’t do this the title is shown left-aligned, possibly followed by one or more (centred) sub-titles. Could the non-excluded titles also be centred please.

“Exclude title from Display” module has now been ported from D7 to D8

Fixed. Tested (11/3/20 Gareth)

Complete
75 High Brian Phil

In UK&I plain page titles are also not centered.

  Complete
76 High Brian Ken On DEV place pages, the text "Click on map to view parish boundaries in detail” is missing from below the clickable parish maps. Done. Complete
77 High Phil Phil Search engine currently disabled. We have been attacked via solr and iptables need to be set to prevent remote access. Needs help from mythic to discover why this isn't working Finally noticed typing error, etho instead of eth0 Complete
78 High Phil   Map on county pages invisible when not logged in This required an access method to be added to the Place Map block. Complete
80 Important Brian   Brian has requested the location images on DEV place nodes are enlarged to fill the available area between the two list of topic links Ken: Phil tried a quick solution, but had knock-on effects elsewhere.  Solution needs to be investigated further after Go Live.    Fixed - Tested - Brian 15/3/20 Complete
82 Important Gareth Ken On the Node Edit screen, Gareth has suggested it would be a good idea to re-order the Local Task Bar into a more logical order (from the simple editing point of view) from the left such as View/Edit/Revisions/Delete/Clone and the rest as you will. Fixed.  Tested on all node types(Gareth 14/3/20) Complete
84 Important Phil   We need a way to specify responsive image styles on media images added via ckeditor. And more importantly in the code that  shows the map at the top of place pages. These have conflicting requirements and the default cannot satisfy both if we enable responsive images which is the way forward.

Test of centring of images by  Brian 14/3/20

No work done on this task which provides that ability to manage image sizes and a much more flexible of handling the different requirements for responsive images in different screen sizes. For fixed sized images we may want thumbnails in some contexts. For small images we probably want them to grow on small windows but only up to a certain size on wide screens as they do not have sufficient resolution. We need to be able to use drupal responsive image styling which can handle all these things.

Not started
86 Important Brian   The lining up of the second columns in /big/eng/DEV/indexpars is not right. Caused by use of styling in table html tags. Styling added to ckeditor to enable classes to the set on <table> and <td>. Just click the table or column and then select the style from the menu. Completed